Canon's New Patent - The Eyes Have it

I read with great interest about Canon's new patent whereby their camera will actually take a retinal scan of your iris, and produce a biometric signature that the camera will then embed into the photographs, authenticating the images as yours, and aiding in locating who the person is that took the photograph via some form of steganography. Of course, that image, above, of Sen. Obama campaigning, is a nice photo-illustration of the concept, and not how your actual image would look!

I've gone through and dissected their patent application after the jump, and discussed some of the interesting findings.

(Continued after the Jump)

First things first, patent application reading is not for the faint of heart, and it does require some creative thinking when it comes to envisioning how a bunch of lawyers are depicting their new (and supposedly valuable) idea.

I've gone through, and pulled out graphics and text that I think do a good job of making it easier to understand (and get excited about!)

Canon posted this patent application on January 31st, and have proposed some very interesting ideas to help us cure the age-old question of "who took this picture?"

The patent application reads:

FIELD OF THE INVENTION
[0001] This invention relates to an imaging apparatus for electrically taking the image of a subject. More particularly,
the invention relates to an imaging apparatus with which it is possible to protect the copyright of the photographer.

[0004] Against this background, the necessity for imaging apparatus such as digital still cameras and digital video cameras has grown explosively.

The application goes on to say why this is necessary:

[0005] The widespread use of personal computers and the like has made the copying of digital data easier and less expensive. In addition, easier access to the Internet has facilitated and lowered the cost of distributing digital data. As a consequence, even ordinary individuals can now create and distribute copies of digital images easily and inexpensively for purposes beyond private use. Accordingly, though the handling ofdigital images does not pose a major problem so long as it involves photography for personal enjoyment, the fact that such image data can be copied and distributed easily by unauthorized individuals has not gone unnoticed by those who circulate digital images as a business.

[0006] Thus, a problem which arises is that protection of the copyright of photographers, ctc., is not satisfactory. Means for protecting copyright is strongly desired for digital image data obtained by photography.

[0007] A technique referred to as an "electronic watermark" has undergone extensive research for the purpose of realizing copyright protection of digital image data. This technique is one in which a portion of the data in digital image data or digital audio data is embedded with separate information by superposition in such a manner that the information is rendered insensible or intentionally sensible to a human being, depending upon the particular purpose. When necessary, only a user having the right or qualifications can extract or remove the embedded separate information.

Below, you'll see just where the iris scan hardware would be placed - tucked right at the top of the viewfinder:

Above, is a graphical representation of the interaction between the iris sensor and the camera's CPU. (note, the correlation between the item identified as "113" in both graphics.

Here's how the application describes it:

[0056] An iris sensor 113, which employs an image sensing device such as a CCD, is placed so as to become the conjugate ofthe pupil 216 ofthe photographer's eye situated at a prescribed position in relation to a light-receiving lens 211. Infrared light-emitting diodes 213a, 213b illuminate the vicinity ofthe photographer's pupil. An LCD 214 inside the viewfinder, which is placed at a position where it can be observed by the photographer at the same time that the photographer observes the finder image, displays various settings information, etc. A high-luminance light-emitting diode 215 illuminates a prescribed area on the focusing plate 206.

[0057] Light from the iris of the photographer is reflected by the spectroscope 210a and has its image formed by the light-receiving lens 211 on the iris sensor 113, whereby the iris image is obtained.

Below is the flowchart for how the camera would scan your iris, while you looked through the viewfinder.

The application includes this description of the process:

[0023] First, the eyeball image of the person is acquired by controlling illumination and focus (1101). When the eyeball image is obtained, the eyelid and eyelash are detected, the pupil-iris boundary 21 and outer boundary 22 of the iris are detected, as shown in FIG. 12, and a coordinate system is set up upon dividing the eyeball into areas 23 referred to as analysis bands (1102).

[0024] Next, image analysis (1103), which mainly entails extracting a change in shading of the analysis bands, is performed, and coding is performed based upon the result of analysis (1104). The personal authentication code generated by coding is expressed by a fixed-length array of bits indicated by "l ?s and "O"s.

The patent goes on to say that you can register up to five iris scans, so you, and four other people can use the same camera and simply select which person is using the camera, or, and it's not clear, perhaps it would actually scan your eye and identify you.

Below, is one of the ways that the biometric information is written to the image. While this one illustrates registration selection and biometric data being written to the file when the card is removed, there are other variations, like when you power off (as it does cleaning the sensor now). Canon's concern was the delay in the data write if this biometric data was applied when the files were being written, as that could delay capture times and buffer issues. With camera buffer acceleration, in the future when this becomes real, I'd expect CPU power to be such that it could/would write when the image was captured, but for now, these are the ways - essentially when the camera is idle - when the information is written.

In response to concerns about processing power, and other potential interference with "at the moment of capture" iris scanning, the application states:

[0028] In accordance with this method, an iris pattern or retinal pattern is extracted from an eyeball image acquired at substantially the same time the image of a subject is taken, and the extracted pattern is embedded in the photographic image. As a result, the photographic image and the photographer information are placed in one-to-one correspondence and there is no way for a third party to intervene. This method is effective in that it affords a high reliability as far as copyright information is concerned.

[0029] However, the above method necessitates the task of acquiring the eyeball image at approximately the same time
that the image of the subject is taken. There are also caseswhere the method necessitates the additional task of extracting
the iris pattern or retinal pattern from the eyeball image and converting this pattern to a personal authentication code
by coding means that relies upon image processing. In a digital image sensing device such as a digital still camera, such a task coincides, sequentially speaking, with the timing at which maximum load is imposed upon processing of the subject image at the time of photography. When eyeball image processing is executed along with subject-image processing, therefore, the overall processing requires a great amount oftime. This means that the photographer must wait a while before the next photo can be taken, resulting in possible loss of photographic opportunities.

[0030] The imposition of a heavy processing load in this fashion is not limited to a personal authentication method that uses an iris or retinal pattern but is a common problem also in other biometric personal authentication methods that subject personal biological differences to authentication coding by image processing or the like.

[0031] When the eyeball image of a photographer is acquired every time an image is taken, the photographer's eye may be closed at the moment of acquisition or an eyelash or strand of hair may interfere. In view of the fact that this can happen frequently, the eyeball image may not always be acquired properly. Furthermore, since the pupil of the photographer's eye opens when an image is taken under low illumination, as is the case indoors, the area of the iris pattern becomes comparatively small and it may not be possible to convert the pattern to an accurate personal authentication code.

Below, is where it gets even more interesting:

Instead of using the biometric information from your iris, you could, instead, if you were concerned about image degradation for some reason (i.e. you were seeing degradation, because, from a steganographic standpoint, you should not see the watermark) encode a personal authentication code.

The application states:

[0016] The watermarking technique according to the above description is advantageous in that (1) the embedded data cannot be extracted without the key information used at the time of embedding; (2) since the embed component in the key information is created based upon a random number, the component is not fixed, thereby making it difficult to decode the embedded data; (3) by specially adapting the embed component, data can be embedded so as not to be sensible by a human being; and (4) the degree to which image quality declines can be controlled by changing the amount of alteration.

[0017] An "invisible-data embedding" method through which embedded data is rendered invisible to a human being has been described. As mentioned earlier, however, a "visible-data embedding" method also is available. According to this method, information such as copyright information is embedded in an original image with the intention of being made visible to a human being. This has the effect ofcausing a third party to abandon the idea of utilizing an image unjustly. For details relating to a visible-data watermarking technique, see the specification of u.s. Pat. No. 5,530,759 (Japanese Patent Application Laid-Open No. 8-241403).

[0018] Techniques for authenticating specific individuals are being researched extensively from the standpoint of protecting privacy and providing security.

[0019] A number of methods have generally been employed for authenticating individuals. Examples are a method through which only a specific person is verified by a key, card or seal in his or her possession, and a method through which only a specific person is verified by entry of a password or secret code number known only by the person. A fundamental problem with this method is that it is comparatively easy for another person to pose as the specific person by way of theft, counterfeiting or leakage of information, etc.

[0020] Accordingly, a method that has become the focus of attention as an alternative to the above method is a biometric personal authentication method that employs a physical characteristic of a specific person to undergo authentication.

This is very cool because, the PLUS Coalition is working on an artist ID/registry, whereby people could attempt to locate you if an image with your ID/biometric scan was in an image, they could go to this central repository, and find you, thus, no more Orphaned images! While this wouldn't solve old-image issues, it would be an exceptional solution moving forward!

Here's some information about the registry:

The PLUS Registry is a joint effort by and for all industries and communities engaged in creating, distributing, using and preserving images. The Registry is operated cooperatively by a worldwide coalition of organizations representing the interests of photographers, illustrators,artist representatives, stock agencies, ad agencies, designers, magazine publishers, newspaper publishers, web publishers, book publishers, museums, libraries educational institutions and others. While the organization includes all of the major photography associations in the US and many others across 30 countries, PLUS represents all industries. Within this diverse body are organizations with conflicting viewpoints on the proposed orphan works legislation. However, they have joined forces within PLUS to provide a non-partisan solution to the orphan works challenge. While the PLUS Coalition and the PLUS standards provide benefits far exceeding the scope of the orphan works challenge, the standards and in particular the Registry system provide a global, industry-neutral vehicle for the prevention of orphan works, and for connecting potential licensees with rights holders.

The Registry is operated by a non-profit, neutral organization, on a non-profit basis. It is a modular system, initially with three component registries:

Artist & Licensor Registry

Free registration to all artists, copyright owners, licensors. PLUS membership not required. PLUS-ID optional. Look up by name, company name, or PLUS ID number. Note that this is not a portfolio site or marketing vehicle for artists. This is a single, centralized location not operated by any one industry trade association, where people of all industries may search for and locate the contact information for a rights holder or authorized licensor.

License Registry

Registration and lookup of image licenses via license-ID. Allows external,centralized storage of public and private license metadata. Permissions based access to private information. Open public access to information deemed public by licensor. Licensor enters license information directly into registry or via PLUS compliant software, or via ingestion of metadata embedded in image. Licensor is issued a unique identifier. That identifier is added to the metadata of images distributed by licensor. Anyone encountering that image may then access current rights and attribution for that image via a registry lookup, again, either directly from the registry or via PLUS compliant applications. Accommodates license updates. Prevents unauthorized license metadata tampering and avoids unsynchronized embedded license information. License ID may also be embedded via Digimarc watermark as a 2nd level of protection.

Image Registry

Registration and lookup of images via image ID and/or image recognition. Users upload low resolution images and receive unique identifiers, linked to their myPLUS account. Other users may lookup the rights holder and authorized licensors for any image via image-ID lookup. For those images lacking image IDs, image recognition capability will be offered, providing a licensor/rights holder name when a low resolution copy is uploaded by a searcher.

Importantly, the Image Registry does not allow the browsing of images. Any image may be looked up, but there is no browsing the registry. Also, this is not a registry of potentially orphaned works. The latter type of registry will be a magnet for persons seeking to harvest free images that are already tagged as potential orphan works. it will be the worst scenario for rights holders, and should be discouraged. Further, any such registry is very inefficient for small business owners, as it is entirely impractical to monitor such a registry.

Also note that security measures are implemented across all registries to prevent the harvesting of email addresses and contact information by third parties. Also the registry is hosted on geographically separated redundant mirrored servers to maximize uptime.

Timeline for the registry:

The Artist & Licensor Registry is built and will be enter beta testing shortly. The License registry functionality is developed and will be integrated after an appropriate period of testing of the A&L Registry, and after that registry is populated by opt-in ingestion of trade associations members and non-members. The Image registry will be the last component, and will be tested shortly after the activation of the License registry.

Timeline for the camera?

Who knows, but it's an amazing technological advance!

---

Please post your comments by clicking the link below. If you've got questions, please pose them in our Photo Business Forum Flickr Group Discussion Threads.