Wednesday, February 13, 2008

Canon's New Patent - The Eyes Have it

I read with great interest about Canon's new patent whereby their camera will actually take a retinal scan of your iris, and produce a biometric signature that the camera will then embed into the photographs, authenticating the images as yours, and aiding in locating who the person is that took the photograph via some form of steganography. Of course, that image, above, of Sen. Obama campaigning, is a nice photo-illustration of the concept, and not how your actual image would look!

I've gone through and dissected their patent application after the jump, and discussed some of the interesting findings.

(Continued after the Jump)

First things first, patent application reading is not for the faint of heart, and it does require some creative thinking when it comes to envisioning how a bunch of lawyers are depicting their new (and supposedly valuable) idea.

I've gone through, and pulled out graphics and text that I think do a good job of making it easier to understand (and get excited about!)

Canon posted this patent application on January 31st, and have proposed some very interesting ideas to help us cure the age-old question of "who took this picture?"

The patent application reads:

[0001] This invention relates to an imaging apparatus for electrically taking the image of a subject. More particularly,
the invention relates to an imaging apparatus with which it is possible to protect the copyright of the photographer.

[0004] Against this background, the necessity for imaging apparatus such as digital still cameras and digital video cameras has grown explosively.
The application goes on to say why this is necessary:
[0005] The widespread use of personal computers and the like has made the copying of digital data easier and less expensive. In addition, easier access to the Internet has facilitated and lowered the cost of distributing digital data. As a consequence, even ordinary individuals can now create and distribute copies of digital images easily and inexpensively for purposes beyond private use. Accordingly, though the handling ofdigital images does not pose a major problem so long as it involves photography for personal enjoyment, the fact that such image data can be copied and distributed easily by unauthorized individuals has not gone unnoticed by those who circulate digital images as a business.

[0006] Thus, a problem which arises is that protection of the copyright of photographers, ctc., is not satisfactory. Means for protecting copyright is strongly desired for digital image data obtained by photography.

[0007] A technique referred to as an "electronic watermark" has undergone extensive research for the purpose of realizing copyright protection of digital image data. This technique is one in which a portion of the data in digital image data or digital audio data is embedded with separate information by superposition in such a manner that the information is rendered insensible or intentionally sensible to a human being, depending upon the particular purpose. When necessary, only a user having the right or qualifications can extract or remove the embedded separate information.
Below, you'll see just where the iris scan hardware would be placed - tucked right at the top of the viewfinder:

Above, is a graphical representation of the interaction between the iris sensor and the camera's CPU. (note, the correlation between the item identified as "113" in both graphics.

Here's how the application describes it:
[0056] An iris sensor 113, which employs an image sensing device such as a CCD, is placed so as to become the conjugate ofthe pupil 216 ofthe photographer's eye situated at a prescribed position in relation to a light-receiving lens 211. Infrared light-emitting diodes 213a, 213b illuminate the vicinity ofthe photographer's pupil. An LCD 214 inside the viewfinder, which is placed at a position where it can be observed by the photographer at the same time that the photographer observes the finder image, displays various settings information, etc. A high-luminance light-emitting diode 215 illuminates a prescribed area on the focusing plate 206.

[0057] Light from the iris of the photographer is reflected by the spectroscope 210a and has its image formed by the light-receiving lens 211 on the iris sensor 113, whereby the iris image is obtained.
Below is the flowchart for how the camera would scan your iris, while you looked through the viewfinder.

The application includes this description of the process:
[0023] First, the eyeball image of the person is acquired by controlling illumination and focus (1101). When the eyeball image is obtained, the eyelid and eyelash are detected, the pupil-iris boundary 21 and outer boundary 22 of the iris are detected, as shown in FIG. 12, and a coordinate system is set up upon dividing the eyeball into areas 23 referred to as analysis bands (1102).

[0024] Next, image analysis (1103), which mainly entails extracting a change in shading of the analysis bands, is performed, and coding is performed based upon the result of analysis (1104). The personal authentication code generated by coding is expressed by a fixed-length array of bits indicated by "l ?s and "O"s.
The patent goes on to say that you can register up to five iris scans, so you, and four other people can use the same camera and simply select which person is using the camera, or, and it's not clear, perhaps it would actually scan your eye and identify you.

Below, is one of the ways that the biometric information is written to the image. While this one illustrates registration selection and biometric data being written to the file when the card is removed, there are other variations, like when you power off (as it does cleaning the sensor now). Canon's concern was the delay in the data write if this biometric data was applied when the files were being written, as that could delay capture times and buffer issues. With camera buffer acceleration, in the future when this becomes real, I'd expect CPU power to be such that it could/would write when the image was captured, but for now, these are the ways - essentially when the camera is idle - when the information is written.

In response to concerns about processing power, and other potential interference with "at the moment of capture" iris scanning, the application states:
[0028] In accordance with this method, an iris pattern or retinal pattern is extracted from an eyeball image acquired at substantially the same time the image of a subject is taken, and the extracted pattern is embedded in the photographic image. As a result, the photographic image and the photographer information are placed in one-to-one correspondence and there is no way for a third party to intervene. This method is effective in that it affords a high reliability as far as copyright information is concerned.

[0029] However, the above method necessitates the task of acquiring the eyeball image at approximately the same time
that the image of the subject is taken. There are also caseswhere the method necessitates the additional task of extracting
the iris pattern or retinal pattern from the eyeball image and converting this pattern to a personal authentication code
by coding means that relies upon image processing. In a digital image sensing device such as a digital still camera, such a task coincides, sequentially speaking, with the timing at which maximum load is imposed upon processing of the subject image at the time of photography. When eyeball image processing is executed along with subject-image processing, therefore, the overall processing requires a great amount oftime. This means that the photographer must wait a while before the next photo can be taken, resulting in possible loss of photographic opportunities.

[0030] The imposition of a heavy processing load in this fashion is not limited to a personal authentication method that uses an iris or retinal pattern but is a common problem also in other biometric personal authentication methods that subject personal biological differences to authentication coding by image processing or the like.

[0031] When the eyeball image of a photographer is acquired every time an image is taken, the photographer's eye may be closed at the moment of acquisition or an eyelash or strand of hair may interfere. In view of the fact that this can happen frequently, the eyeball image may not always be acquired properly. Furthermore, since the pupil of the photographer's eye opens when an image is taken under low illumination, as is the case indoors, the area of the iris pattern becomes comparatively small and it may not be possible to convert the pattern to an accurate personal authentication code.
Below, is where it gets even more interesting:

Instead of using the biometric information from your iris, you could, instead, if you were concerned about image degradation for some reason (i.e. you were seeing degradation, because, from a steganographic standpoint, you should not see the watermark) encode a personal authentication code.

The application states:
[0016] The watermarking technique according to the above description is advantageous in that (1) the embedded data cannot be extracted without the key information used at the time of embedding; (2) since the embed component in the key information is created based upon a random number, the component is not fixed, thereby making it difficult to decode the embedded data; (3) by specially adapting the embed component, data can be embedded so as not to be sensible by a human being; and (4) the degree to which image quality declines can be controlled by changing the amount of alteration.

[0017] An "invisible-data embedding" method through which embedded data is rendered invisible to a human being has been described. As mentioned earlier, however, a "visible-data embedding" method also is available. According to this method, information such as copyright information is embedded in an original image with the intention of being made visible to a human being. This has the effect ofcausing a third party to abandon the idea of utilizing an image unjustly. For details relating to a visible-data watermarking technique, see the specification of u.s. Pat. No. 5,530,759 (Japanese Patent Application Laid-Open No. 8-241403).

[0018] Techniques for authenticating specific individuals are being researched extensively from the standpoint of protecting privacy and providing security.

[0019] A number of methods have generally been employed for authenticating individuals. Examples are a method through which only a specific person is verified by a key, card or seal in his or her possession, and a method through which only a specific person is verified by entry of a password or secret code number known only by the person. A fundamental problem with this method is that it is comparatively easy for another person to pose as the specific person by way of theft, counterfeiting or leakage of information, etc.

[0020] Accordingly, a method that has become the focus of attention as an alternative to the above method is a biometric personal authentication method that employs a physical characteristic of a specific person to undergo authentication.
This is very cool because, the PLUS Coalition is working on an artist ID/registry, whereby people could attempt to locate you if an image with your ID/biometric scan was in an image, they could go to this central repository, and find you, thus, no more Orphaned images! While this wouldn't solve old-image issues, it would be an exceptional solution moving forward!

Here's some information about the registry:

The PLUS Registry is a joint effort by and for all industries and communities engaged in creating, distributing, using and preserving images. The Registry is operated cooperatively by a worldwide coalition of organizations representing the interests of photographers, illustrators,artist representatives, stock agencies, ad agencies, designers, magazine publishers, newspaper publishers, web publishers, book publishers, museums, libraries educational institutions and others. While the organization includes all of the major photography associations in the US and many others across 30 countries, PLUS represents all industries. Within this diverse body are organizations with conflicting viewpoints on the proposed orphan works legislation. However, they have joined forces within PLUS to provide a non-partisan solution to the orphan works challenge. While the PLUS Coalition and the PLUS standards provide benefits far exceeding the scope of the orphan works challenge, the standards and in particular the Registry system provide a global, industry-neutral vehicle for the prevention of orphan works, and for connecting potential licensees with rights holders.

The Registry is operated by a non-profit, neutral organization, on a non-profit basis. It is a modular system, initially with three component registries:

Artist & Licensor Registry
Free registration to all artists, copyright owners, licensors. PLUS membership not required. PLUS-ID optional. Look up by name, company name, or PLUS ID number. Note that this is not a portfolio site or marketing vehicle for artists. This is a single, centralized location not operated by any one industry trade association, where people of all industries may search for and locate the contact information for a rights holder or authorized licensor.
License Registry
Registration and lookup of image licenses via license-ID. Allows external,centralized storage of public and private license metadata. Permissions based access to private information. Open public access to information deemed public by licensor. Licensor enters license information directly into registry or via PLUS compliant software, or via ingestion of metadata embedded in image. Licensor is issued a unique identifier. That identifier is added to the metadata of images distributed by licensor. Anyone encountering that image may then access current rights and attribution for that image via a registry lookup, again, either directly from the registry or via PLUS compliant applications. Accommodates license updates. Prevents unauthorized license metadata tampering and avoids unsynchronized embedded license information. License ID may also be embedded via Digimarc watermark as a 2nd level of protection.
Image Registry
Registration and lookup of images via image ID and/or image recognition. Users upload low resolution images and receive unique identifiers, linked to their myPLUS account. Other users may lookup the rights holder and authorized licensors for any image via image-ID lookup. For those images lacking image IDs, image recognition capability will be offered, providing a licensor/rights holder name when a low resolution copy is uploaded by a searcher.
Importantly, the Image Registry does not allow the browsing of images. Any image may be looked up, but there is no browsing the registry. Also, this is not a registry of potentially orphaned works. The latter type of registry will be a magnet for persons seeking to harvest free images that are already tagged as potential orphan works. it will be the worst scenario for rights holders, and should be discouraged. Further, any such registry is very inefficient for small business owners, as it is entirely impractical to monitor such a registry.

Also note that security measures are implemented across all registries to prevent the harvesting of email addresses and contact information by third parties. Also the registry is hosted on geographically separated redundant mirrored servers to maximize uptime.

Timeline for the registry:

The Artist & Licensor Registry is built and will be enter beta testing shortly. The License registry functionality is developed and will be integrated after an appropriate period of testing of the A&L Registry, and after that registry is populated by opt-in ingestion of trade associations members and non-members. The Image registry will be the last component, and will be tested shortly after the activation of the License registry.

Timeline for the camera?

Who knows, but it's an amazing technological advance!

Please post your comments by clicking the link below. If you've got questions, please pose them in our Photo Business Forum Flickr Group Discussion Threads.


Kalmár Nagy András said...

Problem is, now with live-view in most cameras (it's the "got to implement it somehow" feature of the year) people will not always have their eyes to the camera viewfinder.

John Harrington said...

Right. The patent has it scanned beforehand, so it doesn't do it for each image made!


Anonymous said...

John... I caught this as well and talked about this on my blog, but not nearly to the extent you have. You amaze me with the detail you've added here.

The big litmus test with this technology... how well does the embedded information survive a screen capture, resizing and resaving. This is the number one way in which my work is constantly infringed and I go above and beyond to frame, label and watermark my images. In all honesty I fear this great technology will be completely undermined by the most simple of computer functions... the screen capture.


Anonymous said...

Hi. I am faithful addict of you blog for a long time, but I have the urge to say what I feel (ok, maybe and what I feel :) )
I am against biometry in all forms because I think that this could lead to some sort of goverment control. (not exactly a conspiracy theory, but not far away :) )
Besides, it seems to me that main drive for this patent was increasing income through marketing. "With OUR camera you are safe" or soemthing like that.
If Canon had protection from copyright infingment they could easily build a server farm where phtographers could upload photos, with all EXIF data. Google has enough powerfull search engine to search through all that data and find what ever is needed. Artificial inteligence can compare photos easily and find duplicates - there are a lot more better solutions than retina scan. Call me paranoic, but I don't beleive in good intentions when someone is collecting your retina scan. It's to intimate :)))))
Just my 2 cents (or whatever :)
I realy enjoy reading your blog and find it very valuable. Congratulations and keep good work.

Anonymous said...

What about if you loan your camera to someone and that person takes a great photo? Who gets credit?

John Harrington said...

Alex --

There is the option to NOT have your biometry, and instead, just a unique number.

Anonymous - I presume you can turn it off, and also , it will store 5 different users.


Anonymous said...

For lossless image compression, Image scanning I have transformed my image using lifting wavelet transformation, now I should do Huffman coding on transformed matrix

Anonymous said...

情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,美國aneros,rudeboy,英國rudeboy,英國Rocksoff,德國Fun Factory,Fun Factory,英國甜筒造型按摩座,甜筒造型按摩座,英國Rock Chic ,瑞典 Lelo ,英國Emotional Bliss,英國 E.B,荷蘭 Natural Contours,荷蘭 N C,美國 OhMiBod,美國 OMB,Naughti Nano ,音樂按摩棒,ipod按摩棒,美國 The Screaming O,美國TSO,美國TOPCO,美國Doc Johnson,美國CA Exotic,美國CEN,美國Nasstoy,美國Tonguejoy,英國Je Joue,美國Pipe Dream,美國California Exotic,美國NassToys,美國Vibropod,美國Penthouse,仿真按摩棒,矽膠按摩棒,猛男倒模,真人倒模,仿真倒模,PJUR,Zestra,適趣液,穿戴套具,日本NPG,雙頭龍,FANCARNAL,日本NIPPORI,日本GEL,日本Aqua Style,美國WET,費洛蒙,費洛蒙香水,仿真名器,av女優,打炮,做愛,性愛,口交,吹喇叭,肛交,魔女訓練大師,無線跳蛋,有線跳蛋,震動棒,震動保險套,震動套,TOY-情趣用品,情趣用品網,情趣購物網,成人用品網,情趣用品討論,成人購物網,鎖精套,鎖精環,持久環,持久套,拉珠,逼真按摩棒,名器,超名器,逼真老二,電動自慰,自慰,打手槍,仿真女郎,SM道具,SM,性感內褲,仿真按摩棒,pornograph,hunter系列,h動畫,成人動畫,成人卡通,情色動畫,情色卡通,色情動畫,色情卡通,無修正,禁斷,人妻,極悪調教,姦淫,近親相姦,顏射,盜攝,偷拍,本土自拍,素人自拍,公園露出,街道露出,野外露出,誘姦,迷姦,輪姦,凌辱,痴漢,痴女,素人娘,中出,巨乳,調教,潮吹,av,a片,成人影片,成人影音,線上影片,成人光碟,成人無碼,成人dvd,情色影音,情色影片,情色dvd,情色光碟,航空版,薄碼,色情dvd,色情影音,色情光碟,線上A片,免費A片,A片下載,成人電影,色情電影,TOKYO HOT,SKY ANGEL,一本道,SOD,S1,ALICE JAPAN,皇冠系列,老虎系列,東京熱,亞熱,武士系列,新潮館,情趣用品,約定金生,約定金生,情趣,情趣商品,約定金生,情趣網站,跳蛋, 約定金生,按摩棒,充氣娃娃,約定金生,自慰套,G點,性感內衣,約定金生,情趣內衣,約定金生,角色扮演,生日禮物,生日精品,約定金生,自慰,打手槍,約定金生,潮吹,高潮,後庭,約定金生,情色論譠,影片下載,約定金生,遊戲下載,手機鈴聲,約定金生,音樂下載, 約定金生,約定金生,開獎號碼,統一發票號碼,夜市,統一發票對獎,保險套, 約定金生,約定金生,做愛,約定金生,減肥,美容,瘦身,約定金生,當舖,軟體下載,汽車,機車, 約定金生,手機,來電答鈴, 約定金生,週年慶,美食,約定金生,徵信社,網頁設計,網站設計, 約定金生,室內設計, 約定金生,靈異照片,約定金生,同志,約定金生,聊天室,運動彩券,大樂透,約定金生,威力彩,搬家公司,除蟲,偷拍,自拍, 約定金生,無名破解,av女優, 約定金生,小說,約定金生,民宿,大樂透開獎號碼,大樂透中獎號碼,威力彩開獎號碼,約定金生,討論區,痴漢,懷孕, 約定金生,約定金生,美女交友,約定金生,交友,日本av,日本,機票, 約定金生,香水,股市, 約定金生,股市行情, 股市分析,租房子,成人影片,約定金生,免費影片,醫學美容, 約定金生,免費算命,算命,約定金生,姓名配對,姓名學,約定金生,姓名學免費,遊戲, 約定金生,好玩遊戲,好玩遊戲區,約定金生,線上遊戲,新遊戲,漫畫,約定金生,線上漫畫,動畫,成人圖片, 約定金生,桌布,桌布下載,電視節目表, 約定金生,線上電視,約定金生,線上a片,約定金生,線上掃毒,線上翻譯,購物車,約定金生,身分證製造機,身分證產生器,手機,二手車,中古車, 約定金生,約定金生,法拍屋,約定金生,歌詞,音樂,音樂網,火車,房屋,情趣用品,約定金生,情趣,情趣商品,情趣網站,跳蛋,約定金生,按摩棒,充氣娃娃,自慰套, 約定金生, G點,性感內衣,約定金生,情趣內衣,約定金生,角色扮演,生日禮物,精品,禮品,約定金生,自慰,打手槍,潮吹,高潮,約定金生,後庭,情色論譠,約定金生,影片下載,約定金生,遊戲下載,手機鈴聲,音樂下載,開獎號碼,統一發票,夜市,保險套,做愛,約定金生,減肥,美容,瘦身,當舖,約定金生,軟體下載,約定金生,汽車,機車,手機,來電答鈴,約定金生,週年慶,美食,徵信社,網頁設計,網站設計,室內設計,靈異照片, 約定金生,同志,聊天室,約定金生,運動彩券,,大樂透,約定金生,威力彩,搬家公司,除蟲,偷拍,自拍, 約定金生,無名破解, av女優,小說,民宿,約定金生,大樂透開獎號碼,大樂透中獎號碼,威力彩開獎號碼,討論區,痴漢, 約定金生,懷孕,約定金生,美女交友,約定金生,交友,日本av ,日本,機票, 約定金生,香水,股市, 約定金生,股市行情,股市分析,租房子,約定金生,成人影片,免費影片,醫學美容,免費算命,算命, 約定金生,姓名配對,姓名學, 約定金生,姓名學免費,遊戲,約定金生,好玩遊戲,約定金生,好玩遊戲區,線上遊戲,新遊戲,漫畫,線上漫畫,動畫,成人圖片,桌布,約定金生,桌布下載,電視節目表,線上電視, 約定金生,線上a片,線上a片,線上翻譯, 約定金生,購物車,身分證製造機,約定金生,身分證產生器,手機,二手車,中古車,法拍屋,歌詞,音樂,音樂網, 約定金生,借錢,房屋,街頭籃球,找工作,旅行社,約定金生,六合彩,整型,水噹噹,貸款,貸款,信用貸款,宜蘭民宿,花蓮民宿,未婚聯誼,網路購物,珠海,下川島,常平,珠海,澳門機票,香港機票,婚友,婚友社,未婚聯誼,交友,婚友,婚友社,單身聯誼,未婚聯誼,未婚聯誼,婚友社,婚友,婚友社,單身聯誼,婚友,未婚聯誼,婚友社,未婚聯誼,單身聯誼,單身聯誼,婚友,單身聯誼,未婚聯誼,婚友,交友,交友,婚友社,婚友社,婚友社,大陸新娘,大陸新娘,大陸新娘,越南新娘,越南新娘,外籍新娘,外籍新娘,台中坐月子中心,搬家公司,搬家,搬家,搬家公司,線上客服,網頁設計,線上客服,網頁設計,網頁設計,土地貸款,免費資源,電腦教學,wordpress,人工植牙,關鍵字,關鍵字,seo,seo,網路排名,自然排序,網路排名軟體,

Anonymous said...

^^ nice blog!! ^@^

徵信, 徵信, 徵信, 徵信社, 徵信社, 徵信社, 感情挽回, 婚姻挽回, 挽回婚姻, 挽回感情, 徵信, 徵信社, 徵信, 徵信, 捉姦, 徵信公司, 通姦, 通姦罪, 抓姦, 抓猴, 捉猴, 捉姦, 監聽, 調查跟蹤, 反跟蹤, 外遇問題, 徵信, 捉姦, 女人徵信, 女子徵信, 外遇問題, 女子徵信, 徵信社, 外遇, 徵信公司, 徵信網, 外遇蒐證, 抓姦, 抓猴, 捉猴, 調查跟蹤, 反跟蹤, 感情挽回, 挽回感情, 婚姻挽回, 挽回婚姻, 外遇沖開, 抓姦, 女子徵信, 外遇蒐證, 外遇, 通姦, 通姦罪, 贍養費, 徵信, 徵信社, 抓姦, 徵信社, 徵信, 徵信公司, 徵信社, 徵信, 徵信公司, 徵信社, 徵信公司, 女人徵信, 外遇

徵信, 徵信網, 徵信社, 徵信網, 外遇, 徵信, 徵信社, 抓姦, 徵信, 女人徵信, 徵信社, 女人徵信社, 外遇, 抓姦, 徵信公司, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 徵信社, 女人徵信社, 徵信社, 徵信, 徵信社, 徵信, 女子徵信社, 女子徵信社, 女子徵信社, 女子徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社,

Anonymous said...

^^ nice blog!! thanks a lot! ^^

徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 離婚, 離婚,

徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 征信, 征信, 徵信, 徵信社, 徵信, 徵信社, 征信, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信, 徵信社, 徵信社, 徵信社, 徵信

Newer Post Older Post