Expeed

{{short description|Nikon media processors}} thumb|right|Expeed logo The '''Nikon Expeed''' image/video processors (often styled ''EXPEED'') are media processors for Nikon's digital cameras.

They perform a large number of tasks:

* Bayer filtering * demosaicing * image sensor corrections/dark-frame subtraction * image noise reduction * image sharpening * image scaling * gamma correction * image enhancement/Active D-Lighting * colorspace conversion * chroma subsampling * framerate conversion * lens distortion/chromatic aberration correction * image compression/JPEG encoding * video compression * display/video interface driving * digital image editing * face detection * audio processing/compression/encoding and * computer data storage/data transmission.

Expeed's multi-processor system on a chip solution integrates an image processor in multi-core processor architecture, with each single processor-core able to compute many instructions/operations in parallel. Storage and display interfaces and other modules are added and a digital signal processor (DSP) increases the number of simultaneous computations. On-chip 32-bit microcontroller initiates and controls the operation and data transfers of all processors, modules, interfaces and can be seen as the main control unit of the camera.

In each generation Nikon uses different versions for its professional and consumer DSLRs / MILCs, whereas its compact cameras use completely different architectures. This is different from for example Canons DIGIC: its professional DSLRs double the processors of its consumer DSLR series. The ''Expeed'' is an application-specific integrated circuit (ASIC) built by Socionext specifically for Nikon designs according to Nikon specifications. [[File:Nikon D90 - board 0 - Nikon Expeed EI-149-1769.jpg|thumb|250x250px|A Nikon Expeed, including an image/video processor]]

==Technology== The ''Nikon Expeed'' is based on the Socionext Milbeaut imaging processors with 16-bit per pixel<ref>{{cite web |url=http://www.nikon.com/news/2007/0823_06.htm |title=Nikon: Nikon's original digital image-processing concept "Expeed" |access-date=2012-03-14 |archive-url=https://web.archive.org/web/20150402104638/http://www.nikon.com/news/2007/0823_06.htm |archive-date=2015-04-02 |url-status=dead }}</ref> multi-core FR-V processor architecture, using a highly parallel pipelined architecture which allows efficient hardware use, increasing throughput and reducing power consumption.

Each core uses an eight-way 256-bit very long instruction word (VLIW, MIMD) and is organized in a four-unit superscalar pipelined architecture (Integer (ALU)-, Floating-point- and two media-processor-units) giving a peak performance of up to 28 instructions per clock cycle and core. Due to the used four-way single instruction, multiple data (SIMD) vector processor units, data is processed with up to 112 data operations per cycle and core.<ref name="frvpap03">{{cite web |url=http://www.fujitsu.com/downloads/MAG/vol42-2/paper03.pdf |title=Fujitsu: FR-V single-chip multicore processor |access-date=2012-03-24 |archive-url=https://web.archive.org/web/20150402150434/http://www.fujitsu.com/downloads/MAG/vol42-2/paper03.pdf |archive-date=2015-04-02 |url-status=dead }}</ref>

An on-chip 32-bit Fujitsu FR RISC micro-controller core is used to initiate and control all processors, modules and interfaces.<ref name="fujimil">{{Cite web |url=http://www.fujitsu.com/global/services/microelectronics/product/assp/milbeaut/ |title=Fujitsu: Milbeaut Imaging Processors |access-date=2012-02-27 |archive-date=2014-02-07 |archive-url=https://web.archive.org/web/20140207120718/http://www.fujitsu.com/global/services/microelectronics/product/assp/milbeaut/ |url-status=live }}</ref><ref name="fujipdf3">{{Cite web |url=http://www.fujitsu.com/downloads/EDG/binary/pdf/find/24-1e/3.pdf |title=Fujitsu: Image processing system for digital cameras: Milbeaut M-4 |access-date=2012-03-14 |archive-date=2015-09-24 |archive-url=https://web.archive.org/web/20150924020507/http://www.fujitsu.com/downloads/EDG/binary/pdf/find/24-1e/3.pdf |url-status=dead }}</ref><ref name="nhack">{{cite web |url=http://nikonhacker.com/wiki/Camera_Matrix |title=Nikon Hacker: Camera Matrix |access-date=2012-12-12 |archive-url=https://web.archive.org/web/20150403062946/https://nikonhacker.com/wiki/Camera_Matrix |archive-date=2015-04-03 |url-status=dead }}</ref><ref name="photop">{{Cite web |url=http://photo-parts.com.ua/parts/dbphp.php?part=Nikon |title=Photo-Parts: Nikon DSC models + components search result |access-date=2012-02-27 |archive-date=2016-03-04 |archive-url=https://web.archive.org/web/20160304075552/http://photo-parts.com.ua/parts/dbphp.php?part=Nikon |url-status=live }}</ref><ref>{{Cite web |url=http://www.fujitsu.com/global/news/pr/archives/month/2005/20050207-01.html |title=Fujitsu Develops Multi-core Processor for High-Performance Digital Consumer Products |access-date=2012-02-27 |archive-date=2013-12-14 |archive-url=https://web.archive.org/web/20131214162840/http://www.fujitsu.com/global/news/pr/archives/month/2005/20050207-01.html |url-status=live }}</ref> The ''Expeed'' versions designated EI-14x and the ''Expeed'' 2 and 3 additionally include a HD video codec engine (FR-V based) and a 16-bit DSP with separate on-chip 4-block Harvard RAM which is usable for example for additional image- and audio-processing. The ''Expeed 3 (FR)'' (EI-158/175) is based on an improved ''Expeed 2'' EI-154 with greatly increased processing speed.

A new architecture in the ''Expeed 3 (ARM)'' offers a highly increased speed in its image processor (with even two pipelines on the EI-160), its H.264 video encoder and is controlled by a dual-core ARM architecture microcontroller replacing the Fujitsu FR.<ref name=nexp3 >{{cite web |url=http://www.nikonusa.com/Learn-And-Explore/Photography-Glossary/A/1/Expeed3-D-SLR-series.html |title=Nikon: Expeed3 (D-SLR series) |access-date=2012-03-21 |archive-url=https://web.archive.org/web/20120127002321/http://www.nikonusa.com/Learn-And-Explore/Photography-Glossary/A/1/EXPEED3-D-SLR-series.html |archive-date=2012-01-27 |url-status=dead }}</ref><ref name=f6 >{{Cite web |url=http://jp.fujitsu.com/group/fsl/en/release/20110208.html |title=Fujitsu Releases 6th Generation of Milbeaut Imaging Processors |access-date=2012-02-27 |archive-date=2015-04-02 |archive-url=https://web.archive.org/web/20150402154414/http://jp.fujitsu.com/group/fsl/en/release/20110208.html |url-status=dead }}</ref>

[[File:CMOS Image Sensor Mechanism Illustration.svg|thumb|CMOS Image Sensor with column and row decoders (resembling DRAM decoders) interfacing the electric charge of the photo diodes. Nikon's CMOS sensors have additionally patented integrated charge amplifiers.<ref>[http://imaging.nikon.com/history/scenes/24/index.htm Nikon FX-Format CMOS Sensor] {{Webarchive|url=https://web.archive.org/web/20130917004531/http://imaging.nikon.com/history/scenes/24/index.htm |date=2013-09-17 }} Nikon</ref> ]]

===Image sensor interface=== CMOS/CCD image sensors are connected with additional external ICs, Nikon drivers, with the exception of the Nikon D3100.<ref name=nhack/> This is done by a mixed analog/digital interface which controls the sensor digitally, but receives analog signals with parallel 14-bit analog-to-digital (A/D) converters. The ''Expeed'' variants EI-142 and EI-158 use Nikon ASICs to connect all full-frame (FX) digital SLR sensors and additionally the Nikon D300/D300s with 12 simultaneous, parallel analog signal readout channels. Mainly due to a larger settling time it allows improved conversion accuracy compared to the four channels in the previous Nikon D2X / D2Xs, Nikon D200 or the Canon EOS 5D Mark II.<ref>{{Cite web |url=http://ces.cnet.com/8301-33371_1-57352486/nikon-d4-arrives-after-leaks-turn-into-flood/ |title=CNET: Nikon D4 arrives after leaks turn into flood |access-date=2012-03-22 |archive-date=2012-01-06 |archive-url=https://web.archive.org/web/20120106222952/http://ces.cnet.com/8301-33371_1-57352486/nikon-d4-arrives-after-leaks-turn-into-flood/ |url-status=live }}</ref><ref>{{Cite web |url=http://www.nphotomag.com/2012/02/07/nikon-d800-gets-official-specs-release-date-and-price/ |title=NPhoto: Nikon D800 gets official: specs, release date and price |access-date=2012-03-22 |archive-date=2012-04-19 |archive-url=https://web.archive.org/web/20120419023059/http://www.nphotomag.com/2012/02/07/nikon-d800-gets-official-specs-release-date-and-price/ |url-status=live }}</ref><ref>[http://scrollingnews.blogspot.de/2012/03/canon-5d-mark-iii-vs-5d-mark-ii-vs.html Canon 5D Mark III vs 5D Mark II Vs NikonD800 VsNikon D700]{{Dead link|date=December 2019 |bot=InternetArchiveBot |fix-attempted=yes }}</ref> six-channel readout is supported by the EI-149 and EI-154 used on the D90, D5000, D7000 and D5100.

The D3100 uses an Analog Devices sensor interface with integrated analog-to-digital converters. The result is a dynamic range ''only'' at the level of competitors like the (higher priced) Canon EOS 600D;<ref>{{Cite web |url=http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/%28appareil1%29/801%7C0/%28brand%29/Nikon/%28appareil2%29/692%7C0/%28brand2%29/Canon/%28appareil3%29/664%7C0/%28brand3%29/Nikon |title=Dxomark: Nikon D3200 and others: Compare cameras side by side |access-date=2021-12-28 |archive-date=2013-10-14 |archive-url=https://web.archive.org/web/20131014111548/http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/801%7C0/(brand)/Nikon/(appareil2)/692%7C0/(brand2)/Canon/(appareil3)/664%7C0/(brand3)/Nikon |url-status=live }}</ref> lower than other Nikon DSLRs with the same Expeed 2 variant.

The ''Expeed'' EI-15x and EI-17x A/D converters allow an increased image sensor readout clock frequency with improved A/D converter accuracy, especially when using 14-bit sampling. ''Expeed'' A/D converters used for EI-149 or all EI-142 need considerably reduced clock rates (1.8 fps on Nikon D3X) for higher accuracy, limiting for example the D3s dynamic range at low ISOs.<ref name=comd8 >{{Cite web |url=http://home.comcast.net/~NikonD70/Charts/PDR.htm#D4,D4(e),D800,D800(e) |title=Sensor Analysis Primer – Engineering and Photographic Dynamic Range by Bill Claff |access-date=2012-03-24 |archive-date=2012-04-06 |archive-url=https://web.archive.org/web/20120406195935/http://home.comcast.net/~NikonD70/Charts/PDR.htm#D4,D4(e),D800,D800(e) |url-status=live }}</ref><ref name=test8 >{{Cite web |url=http://actionphotosbymarianne.com/TestIm/ISO800snr.gif |title=ActionPhotos by Marianne: ISO 800 SNR: Nikon D800 compared |access-date=2012-03-24 |archive-date=2013-12-31 |archive-url=https://web.archive.org/web/20131231001045/http://actionphotosbymarianne.com/TestIm/ISO800snr.gif |url-status=live }}</ref><ref name=dxorat >[http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Ratings DXOmark Sensor ratings] {{Webarchive|url=https://web.archive.org/web/20120321161023/http://www.dxomark.com/index.php/Cameras/Camera-Sensor-Ratings |date=2012-03-21 }} (needs Flash)</ref>

The ''Expeed 3 (ARM)'', first used in the Nikon 1 series, connects a data stream with 24 digital channels (bus), using A/D converters integrated on the image sensor chip.<ref name=ex3 >{{Cite web |url=http://www.imaging-resource.com/NEWS/1316730752.html |title=Imaging resource: Exclusive: Interview with Masahiro Suzuki, General Manager R&D, Nikon Imaging Company |access-date=2012-03-22 |archive-date=2012-05-17 |archive-url=https://web.archive.org/web/20120517165110/http://www.imaging-resource.com/NEWS/1316730752.html |url-status=live }}</ref>

[[File:Group of pictures illustration.jpg|thumb|Example of a Group Of Pictures showing I-, P- and the B-frames additionally used by the ''Expeed 2'' and higher for MPEG encoding. "All-I" has no P- and B-frames similar to Motion JPEG: Frames are separately processed, lowering redundancy detection and compression.]]

===Video processor=== The first variant, EI-14x originally included a video encoder capable of processing VGA resolution with 30 frames per second and MPEG-4 encoding.<ref name=fujipdf3/> The software based video processor realized with FR-V processors enabled a reprogramming:<ref name=frvpap03/><ref>[http://www.fujitsu.com/ph/news/pr/fmal_20040319.html Fujitsu Releases Three New "FR-V Family" Processors for Media Processing] {{Webarchive|url=https://web.archive.org/web/20081201233323/http://www.fujitsu.com/ph/news/pr/fmal_20040319.html |date=2008-12-01 }} Fujitsu</ref><ref>{{Cite web |url=http://contech.suv.ac.kr/contech/courses/10h2microprocessordesign/IntroducingTheFR500EmbeddedMicroprocessor.pdf |title=Introducing The FR500 Embedded Microprocessor |access-date=2012-11-11 |archive-date=2015-06-10 |archive-url=https://web.archive.org/web/20150610193515/http://contech.suv.ac.kr/contech/courses/10h2microprocessordesign/IntroducingTheFR500EmbeddedMicroprocessor.pdf |url-status=live }}</ref> By using Motion JPEG encoding with 24p frame rate, Nikon achieved 720p HD video resolution. The advantages are easy JPEG image extraction, no motion compensation artifacts and low processing power enabling higher resolution, and the disadvantage is a larger file size, nearly reaching the 2 GB limit (for full compatibility) in 5 minutes. The Nikon D90 was the first DSLR with video recording capabilities.

The ''Expeed 2'' (variant EI-154) greatly expanded the capabilities by its 1080p H.264/MPEG-4 AVC HD video encoder. It also offers an increased image sensor analog signal readout clock rate, reducing rolling shutter.<ref name=comd8/><ref name=test8/><ref name=dxorat/><ref name=imagd4/>

Compared to competitors from Canon (DIGIC, "IPP" compression{{spaced ndash}}MPEG-2<ref>[http://www.drdobbs.com/tools/mpeg-2-and-video-compression/201203059 MPEG-2 and Video Compression] {{Webarchive|url=https://web.archive.org/web/20141223144016/http://www.drdobbs.com/tools/mpeg-2-and-video-compression/201203059 |date=2014-12-23 }} Dr.Dobbs</ref> equivalent: Only one previous frame analyzed<ref name=cancomp/><ref>{{Cite web |url=http://techon.nikkeibp.co.jp/english/NEWS_EN/20080922/158348/ |title=Canon Tips Off Enhanced Capabilities of Its New Image Processor |access-date=2012-11-12 |archive-date=2012-02-13 |archive-url=https://web.archive.org/web/20120213135855/http://techon.nikkeibp.co.jp/english/NEWS_EN/20080922/158348/ |url-status=live }}</ref>) the ''Expeed 2'' offers video compression also based on complex B-frames (bi-directional differencing between frames and motion prediction), which has the advantage of higher quality (lower motion blur based on better motion compensation) even with significant higher compression ratio.<ref>[http://imaging.nikon.com/lineup/dslr/d800/features02.htm Nikon D800/D800E: D-Movie shooting functions] {{Webarchive|url=https://web.archive.org/web/20120720002041/http://imaging.nikon.com/lineup/dslr/d800/features02.htm |date=2012-07-20 }} Nikon</ref> This compression requires considerably higher computing power.<ref>[http://documentation.apple.com/en/finalcutpro/usermanual/chapter_C_section_12.html Video Compression] {{Webarchive|url=https://web.archive.org/web/20120826103940/http://documentation.apple.com/en/finalcutpro/usermanual/chapter_C_section_12.html |date=2012-08-26 }} Apple</ref> In 2012 the Canon 5D Mark III introduced a similar compression called "IPB".<ref name=cancomp >[http://learn.usa.canon.com/resources/articles/2012/ipp_ipb_all_i_compare.htmlp Understanding EOS HD Compression Options] {{Webarchive|url=https://web.archive.org/web/20121231070702/http://learn.usa.canon.com/resources/articles/2012/ipp_ipb_all_i_compare.htmlp |date=2012-12-31 }} Canon</ref> Also introduced was "All-I", which uses the simpler I‑frames (coded pictures) without processing any differences between them, but using a low compression (high data rate) and behaves also in other uses like image extraction quite similar to Motion JPEG formerly used by Nikon.<ref>[http://www.cyberscholar.com/canon/camera/index.cfm?page=trainingCenter&product=eos_1d_x_shooting&productPage=mod0301&menu=mod_on3 EOS-1D X: Video Shooting Workflow: New “ALL-I” video compression] {{Webarchive|url=https://web.archive.org/web/20150506094502/http://www.cyberscholar.com/canon/camera/index.cfm?page=trainingCenter&product=eos_1d_x_shooting&productPage=mod0301&menu=mod_on3 |date=2015-05-06 }} Cyberscholar</ref>

The ''Expeed 3 (FR)'' (variants EI-158 and EI-175) offers no significant change, but introduced the first DSLRs to offer uncompressed video output (8bit 4:2:2) over HDMI: Nikon D4, Nikon D800/D800E, Nikon D600, Nikon D7100 and Nikon D5200. The ''Expeed 3 (ARM)'' introduced high-speed video (slow-motion) in its enhanced H.264 HD video engine.

==Variants== Socionext specifies each Milbeaut generation with different numbers of processors. Nikon gives no details, but uses different designated processors in its professional and consumer lines. Although the Milbeaut (''Expeed'') is used in different Nikon designs and by other manufacturers, the software/firmware specifies many of its functions and details and the number of processors or included modules may vary in this ASIC.

===Pre-Expeed=== thumb|right|Nikon EI-137/EI-138 chipset of the Nikon D80

====Early DSLRs==== DSLRs announced before August 2006 do not contain processors named ''Expeed'' (for example Nikon D70/D70s processor: EI-118), although that does not mean that these processors use a different architecture.<ref name=d70h >{{Cite web |url=http://wiki.berndlux.de/doku.php/elektr:nikond70s_hack |title=Nikon D70s Firmware Hack |access-date=2012-02-28 |archive-date=2010-07-25 |archive-url=https://web.archive.org/web/20100725154534/http://wiki.berndlux.de/doku.php/elektr:nikond70s_hack |url-status=live }}</ref> Or the Nikon D200 processor (EI-126) uses the same, but greatly expanded firmware as the D80 (unofficial "Expeed").

===Expeed=== thumb|right|Nikon D90 motherboard with Expeed EI-149 First used in the Nikon D3 and Nikon D300 in 2007, the Expeed was used later in the Nikon D3X, Nikon D700 and Nikon D300s, marked EI-142, and the consumer line variant with reduced processor cores in the Nikon D90 and Nikon D5000, marked EI-149. It is based on a Socionext Milbeaut imaging-processor with 720p Motion JPEG video encoder, DSP and FR-80 (EI-14x versions) core. It uses a 90 nanometer process technology.

====Unofficial Expeed DSLRs==== The EI-137 variant in the Nikon D60 and Nikon D3000{{spaced ndash}}additionally found in the Nikon D40, Nikon D40x and Nikon D80<ref name=photop/>{{spaced ndash}}is based on the older Milbeaut M-3 in 180 nanometer technology (like all former Expeed/Milbeaut since 2001). It includes a FR-71 core with only 12-bit, two-channel image sensor readout, no DSP, slower memory and has a reduced feature set.<ref>{{cite web |url=http://photo.net/equipment/nikon/D80 |title=Photo.net: Nikon D80 Review |access-date=2012-03-23 |archive-url=https://web.archive.org/web/20120513042619/http://photo.net/equipment/nikon/D80 |archive-date=2012-05-13 |url-status=dead }}</ref>

===Expeed 2=== 1080p H.264/MPEG-4 HD video encoder, improved face detection, image noise reduction and image distortion correction<ref>{{Cite web |url=http://www.freepatentsonline.com/7499082.pdf |title=Patent: Distortion correction circuit for generating distortion-corrected image using data for uncorrected image |access-date=2012-03-14 |archive-date=2016-03-03 |archive-url=https://web.archive.org/web/20160303230738/http://www.freepatentsonline.com/7499082.pdf |url-status=live }}</ref> are the main improved features compared to EI-14x ''Expeeds''.<ref name=fujimil/> Used in the Nikon D7000,<ref>{{cite web |url=http://www.chipworks.com/en/technical-competitive-analysis/resources/recent-teardowns/2011/01/teardown-of-the-nikon-d7000-dslr/ |title=Chipworks: Teardown of the Nikon D7000 DSLR |access-date=2012-02-27 |archive-url=https://web.archive.org/web/20110302084019/http://www.chipworks.com/en/technical-competitive-analysis/resources/recent-teardowns/2011/01/teardown-of-the-nikon-d7000-dslr |archive-date=2011-03-02 |url-status=dead }}</ref> Nikon D3100 and Nikon D5100 and Nikon marked EI-154.<ref>{{Cite web |url=http://www.ifixit.com/Teardown/Nikon-D5100-Teardown/5271/1 |title=ifixit: Nikon D5100 Teardown |access-date=2012-02-27 |archive-date=2012-02-03 |archive-url=https://web.archive.org/web/20120203131458/http://www.ifixit.com/Teardown/Nikon-D5100-Teardown/5271/1 |url-status=live }}</ref> Although image sensor readout clock frequency has increased by a factor of 1.75, A/D converter accuracy is improved, especially when using 14-bit. Image processor performance is increased, performing a higher continuous shooting frame rate even when high ISO noise reduction or Active D-Lighting is activated. The ''Expeed'' EI-15x are controlled by an integrated FR-80/FR-81 core. Power consumption is also decreased by the Socionext 65 nm process.

====Expeed 2 (rebranded Expeed 1)==== The Nikon D3s processor<ref>{{Cite web |url=http://www.imaging-resource.com/PRODS/D3S/D3SA.HTM |title=Imaging-resource: Nikon D3S Full Review |access-date=2012-03-14 |archive-date=2012-03-20 |archive-url=https://web.archive.org/web/20120320213607/http://www.imaging-resource.com/PRODS/D3S/D3SA.HTM |url-status=live }}</ref>{{spaced ndash}}although named ''Expeed'' 2{{spaced ndash}}uses nearly the same EI-142 processor as the previous D3/D3X<ref>{{Cite web |url=http://www.imaging-resource.com/PRODS/D3X/D3XA.HTM |title=Imaging-resource: Nikon D3X Full Review |access-date=2012-03-14 |archive-date=2012-04-05 |archive-url=https://web.archive.org/web/20120405015157/http://www.imaging-resource.com/PRODS/D3X/D3XA.HTM |url-status=live }}</ref> featuring for example only 720p Motion JPEG video. It offers the same image sensor interface with identical speed and A/D converter accuracy, limiting the D3s dynamic range at 200 and especially 100 ISOs lower as the D7000/D5100.<ref name=comd8/><ref name=test8/><ref>[http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/628|0/(brand)/Nikon/(appareil2)/485|0/(brand2)/Nikon/(appareil3)/438|0/(brand3)/Nikon#measuretabs-3 DXOmark: Nikon D3s: Comparison dynamic range] {{Webarchive|url=https://web.archive.org/web/20120504023704/http://www.dxomark.com/index.php/Cameras/Compare-Camera-Sensors/Compare-cameras-side-by-side/(appareil1)/628%7C0/(brand)/Nikon/(appareil2)/485%7C0/(brand2)/Nikon/(appareil3)/438%7C0/(brand3)/Nikon#measuretabs-3 |date=2012-05-04 }} (needs Flash)</ref>

====Expeed C2==== Variant used in some Coolpix compact cameras. Cheaper Nikon compact cameras use Sanyo or Zoran Coach image/video processors; both with a completely different technology and different firmware compared to the ''Expeed''.<ref name=photop/>

===Expeed 3===

====Expeed 3 (FR)==== Compared to the previous Expeed 1 (EI-142), it offers the same improvements as the ''Expeed 2'' EI-154 with Socionext 65 nm process, including increased A/D converter accuracy and image sensor analog signal readout clock rate, reducing rolling shutter.<ref name=comd8/><ref name=test8/><ref name=dxorat/><ref name=imagd4 >{{Cite web |url=http://www.imaging-resource.com/PRODS/nikon-d4/nikon-d4A.HTM |title=Imaging-resource: Nikon D4 Hands-on Preview |access-date=2012-03-23 |archive-date=2012-03-10 |archive-url=https://web.archive.org/web/20120310143954/http://www.imaging-resource.com/PRODS/nikon-d4/nikon-d4A.HTM |url-status=live }}</ref> Computing power is highly increased. Marked EI-158, this variant is used by the Nikon D4 and Nikon D800.<ref>{{Cite web |url=http://imaging.nikon.com/lineup/dslr/d4/features01.htm |title=Nikon: D4, 16.2 megapixel resolution and full-frame FX format |access-date=2012-02-27 |archive-date=2012-02-08 |archive-url=https://web.archive.org/web/20120208024615/http://imaging.nikon.com/lineup/dslr/d4/features01.htm |url-status=live }}</ref><ref>{{Cite web |url=http://imaging.nikon.com/lineup/dslr/d800/features01.htm |title=Nikon D800/D800E: Nikon FX-format CMOS sensor with 36.3 effective megapixels |access-date=2012-02-27 |archive-date=2012-07-13 |archive-url=https://web.archive.org/web/20120713155822/http://imaging.nikon.com/lineup/dslr/d800/features01.htm |url-status=dead }}</ref> The EI-158 was the first ''Expeed'' to offer uncompressed video output (8bit 4:2:2) over HDMI.

The Nikon D600 (teardown<ref>[http://www.ifixit.com/Teardown/Nikon+D600+Teardown/10708/1 Nikon D600 Teardown] {{Webarchive|url=https://web.archive.org/web/20121108214109/http://www.ifixit.com/Teardown/Nikon+D600+Teardown/10708/1 |date=2012-11-08 }} ifixit</ref>), Nikon D3200 and Nikon D5200 (teardown<ref>[http://www.chipworks.com/blog/recentteardowns/2013/01/08/inside-the-nikon-d5200-dslr-toshiba-found/ Inside the Nikon D5200 DSLR – Toshiba found!] {{webarchive|url=https://web.archive.org/web/20130110102538/http://www.chipworks.com/blog/recentteardowns/2013/01/08/inside-the-nikon-d5200-dslr-toshiba-found/ |date=2013-01-10 }} Chipworks</ref>) use an ''Expeed 3'' (EI-175, differently marked ML-1131 on D5200), which is, according to Nikon, the same as used for the D4 and D800 series.<ref name=nhack/><ref>[http://imaging.nikon.com/lineup/dslr/d600/features01.htm D600 image quality: Image-processing engine EXPEED 3] {{Webarchive|url=https://web.archive.org/web/20180528112723/http://imaging.nikon.com/lineup/dslr/d600/features01.htm |date=2018-05-28 }} Nikon.com</ref> Its architecture is close to the ''Expeed 2'' variant EI-154 with some improvements like DDR3 memory, and with increased computing power. The D5200 uses a package on package with a 4 Gbit DDR3 SDRAM on top.

====Expeed 3 (ARM)==== In the Nikon 1 series September 2011 Nikon introduced a new largely changed architecture – the main control unit it uses an ARM microcontroller which requires new firmware compared to the totally different Fujitsu FR microcontroller used in all former Milbeaut and ''Expeed'' processors. It is also the first ''Expeed'' using digital image sensor readout – no analog image sensor interface is needed. In the Nikon 1 cameras introduced September 2011 it uses 1 GB fast DDR2 RAM packaged in 2x 4 Gbit chips. Nikon marked EI-160, manufactured in the Socionext 65 nm process.

High-speed dual multi-core image-processing engines with world record (Nikon claim) 600 megapixels per second speed, enhanced H.264 HD video engine and controlled by a dual-core ARM microcontroller are the main improvements.<ref name=f6/><ref name=ex3/><ref>[http://www.nikonusa.com/Learn-And-Explore/Photography-Glossary/E/1/Expeed-3.html Nikon Expeed 3] {{Webarchive|url=https://web.archive.org/web/20120405144126/http://www.nikonusa.com/Learn-And-Explore/Photography-Glossary/E/1/EXPEED-3.html |date=2012-04-05 }} Nikon USA</ref> Its high speed allows the world's fastest speed (Nikon claim) of 60 frames per second (10 fps with full autofocus).

=====Expeed 3A===== The ''Expeed 3A'', a successor to the Expeed 3 EI-160 used in the Nikon 1 series, was first released in the Nikon 1 V2 and mainly features an increased world record image-processing speed of up to 850 megapixels per second.<ref>[http://www.europe-nikon.com/en_GB/product/digital-cameras/nikon-1/nikon-1-v2 Nikon 1 V2 specifications] {{Webarchive|url=https://web.archive.org/web/20121026112955/http://www.europe-nikon.com/en_GB/product/digital-cameras/nikon-1/nikon-1-v2 |date=2012-10-26 }} Nikon Europe</ref> This enables 60 frames per second (15 fps with full autofocus) speed even with the new 14 megapixel image sensor. It is developed exclusively for Nikon 1 cameras.<ref>[http://nikon.com/news/2012/1024_acil_01.htm Advanced Camera with Interchangeable Lenses Nikon 1 V2] {{Webarchive|url=https://web.archive.org/web/20121027000804/http://nikon.com/news/2012/1024_acil_01.htm |date=2012-10-27 }} Nikon</ref>

===Expeed 4=== Expeed 4, introduced on October 17, 2013 in the Nikon 5300, uses a processor with ARM central controller. It is used in the Nikon D810, Nikon D750, Nikon D5300, Nikon D5500, Nikon D5600, Nikon D3300, Nikon D3400, Nikon D3500 and Nikon D7200. It offers full HD (1080p) video capture at 50/60 fps with improved contrast detection autofocus and live preview autofocus. It includes all of the features of the Expeed 3 (FR) and older versions of Expeed, plus it consumes less power.

The Nikon D4S's processor is identical to the Nikon D4, marked EI-158, using its processing power with improved software enabling 1080p video capture at 50/60fps, 11 fps photos with improved autofocus, new noise reduction with image-content variable algorithm (context-adaptive)<ref>[https://www.researchgate.net/profile/Konstantinos_Plataniotis/publication/222393947_Self-adaptive_algorithm_of_impulsive_noise_reduction_in_color_images/file/3deec5154765f0adfc.pdf Self-adaptive algorithm of impulsive noise reduction in color images]</ref><ref>{{Cite web |url=http://telin.ugent.be/~sanja/Papers/CurveletProbShrink_JEI08.pdf |title=Context adaptive image denoising through modeling of curvelet domain statistics |access-date=2014-03-10 |archive-date=2016-03-04 |archive-url=https://web.archive.org/web/20160304110903/http://telin.ugent.be/~sanja/Papers/CurveletProbShrink_JEI08.pdf |url-status=live }}</ref> and other improvements.<ref>[https://www.youtube.com/watch?v=BCmTyCUEsXI Nikon D4S - Product Tour] {{Webarchive|url=https://web.archive.org/web/20160624215544/https://www.youtube.com/watch?v=BCmTyCUEsXI |date=2016-06-24 }} Nikon - Youtube</ref><ref>{{cite web |title=Nikon D4S |url=http://www.nikonusa.com/en/Nikon-Products/Product/Digital-SLR-Cameras/1541/D4S.html |publisher=Nikon |archive-url=https://web.archive.org/web/20140228151522/http://www.nikonusa.com/en/Nikon-Products/Product/Digital-SLR-Cameras/1541/D4S.html |archive-date=2014-02-28 |url-status=dead}}</ref>

====Expeed 4A==== Version 4A was introduced on the Nikon 1 V3 on March 13, 2014. It was also used in the Nikon 1 J4 and Nikon 1 S2.

===Expeed 5=== Nikon announced EXPEED 5 processor engine in their new DX and FX cameras Nikon D500 and Nikon D5 at CES 2016, (Las Vegas, January 5, 2016)<ref>{{cite web |url=http://www.nikonusa.com/en/nikon-products/product/dslr-cameras/d5.html?icid=hp:topbanner:dslr:d5:010516:wwa |title=Nikon D5 | Professional DSLR with 4K UHD Video & More |publisher=Nikonusa.com |date= |accessdate=2021-12-28 |archive-date=2019-04-20 |archive-url=https://web.archive.org/web/20190420164508/https://www.nikonusa.com/en/nikon-products/product/dslr-cameras/d5.html?icid=hp:topbanner:dslr:d5:010516:wwa |url-status=live }}</ref> and also used in the Nikon D7500 and D850.<ref>{{cite web|url=http://www.nikonusa.com/en/nikon-products/product/dslr-cameras/d850.html#tab-ProductDetail-ProductTabs-Overview|title=D850 Full Frame Digital SLR Camera {{!}} Nikon|website=www.nikonusa.com|language=en|access-date=2017-08-28|archive-date=2017-08-28|archive-url=https://web.archive.org/web/20170828121517/http://www.nikonusa.com/en/nikon-products/product/dslr-cameras/d850.html#tab-ProductDetail-ProductTabs-Overview|url-status=live}}</ref>

====Expeed 5A==== The EXPEED 5A image-processing engine is first seen in the Nikon 1 J5, which was announced April 2, 2015.<ref>{{cite web|access-date=2018-08-23|title=Advanced Camera with Interchangeable Lenses Nikon 1 J5|url=http://www.nikon.com/news/2015/0402_acil_01.htm|website=Nikon|archive-date=2018-08-23|archive-url=https://web.archive.org/web/20180823174146/https://www.nikon.com/news/2015/0402_acil_01.htm|url-status=live}}</ref> In the Nikon 1 J5 it is capable of 4k Ultra HD (3840*2160) at 15 fps, Full-HD (1920*1080) at 60 fps, HD (1280*720) at 120 fps, 800 x 296 at 400 fps and 400 x 144 at 1200 fps. It can handle 20MP burst photos at 20 fps with autofocus at each frame, and even 60 fps with autofocus fixed at the first frame, but note that the buffer size is yet unknown and probably very small.

===Expeed 6=== thumb|right|EXPEED 6 logo  Nikon announced the Expeed 6 processor engine on 23 August 2018.<ref name="nikon-z">{{cite web|access-date=2018-08-23|title=Mirrorless Reinvented: Z Series Cameras|url=https://www.nikonusa.com/en/nikon-products/mirrorless-cameras/overview.page|website=Nikon|archive-date=2018-08-23|archive-url=https://web.archive.org/web/20180823105610/https://www.nikonusa.com/en/nikon-products/mirrorless-cameras/overview.page|url-status=live}}</ref> It was first used on the Nikon Z7, and later appeared in the Z6, Z50, D780, D6, Z5, Zfc, and Z30 cameras. It was also used in the Z6II and Z7II, each containing two Expeed 6 processors. This improves autofocus performance and enables 4K video recording at 60 fps.<ref name="nikon-z"/><ref>{{cite web|url=https://www.nikon.com/news/2019/1010_zmount_01.htm|title=Nikon releases the DX-format Nikon Z 50|access-date=2020-01-14|website=Nikon|archive-date=2019-12-18|archive-url=https://web.archive.org/web/20191218041344/https://www.nikon.com/news/2019/1010_zmount_01.htm|url-status=live}}</ref>

===Expeed 7=== thumb|right|EXPEED 7 logo as it appears in promotional material for cameras containing the processor  Nikon announced the Expeed 7 processor engine on 28 October 2021. It was first used on the Nikon Z9, and later appeared in the Z8, Zf, Z6III, Z50II, Z5II, and ZR mirrorless cameras. Expeed 7 has 10 times the image processing speed of its predecessor and is powerful enough for the camera to omit a dedicated autofocus engine.<ref>{{Cite web |title=Z 9 開発者Special Interview vol.1～フラッグシップ機「Z 9」の開発コンセプトと注目の新機能～ {{!}} ニコンイメージング |url=https://www.nikon-image.com/sp/z9_specialinterview/01_z9/ |access-date=2023-10-11 |website=ニコンイメージングジャパン |language=ja}}</ref> It handles complex AF and AE calculations at 120 cycles per second, separately processes dual-streamed data from stacked image sensors, and, based on the camera, enables features like 120 fps still shooting, internal 8k RAW video recording via intoPIX TicoRAW <ref>{{cite web |title=TicoRAW - IntoPIX |url=https://www.intopix.com/ticoraw |access-date=2025-11-17}}</ref> and a blackout-free viewing experience.<ref>{{Cite web |title=Facebook |url=https://www.facebook.com/KenyaNikon/posts/pfbid02zeBZTmEvEM1jVu5ydB77sqLzC8wurx8XXVHmn9oPiopfk981NQNXpKTSU1qwXPEWl |access-date=2024-06-10 |website=www.facebook.com}}</ref>

==Additional (co-)processors== thumb|FR80-series coprocessor in a Nikon D90 As input/output (I/O) processor Nikon uses external 32-bit microcontrollers to connect additional sensors and displays:

* Viewfinder * top-display * flash (Speedlight)/hot shoe * shutter/aperture motors * metering/autofocus sensors and * lens/battery grip/battery control.

Used variants are the Fujitsu FR in the ''Expeed'' EI-14x series, which changed to MIPS architecture in the ''Expeed'' EI-15x and EI-17x series. The professional series uses two or more Hitachi/Renesas H8SX controllers. Former DSLRs used H8S microcontrollers.<ref>[http://resource.renesas.com/lib/eng/edge/11/special02.html Nikon Corporation using Renesas flash microcomputers] {{Webarchive|url=https://web.archive.org/web/20121217053530/http://resource.renesas.com/lib/eng/edge/11/special02.html |date=2012-12-17 }} Renesas</ref>

The ARM-based ''Expeed'' in the Nikon 1 series with its dual ARM core needs no additional I/O processors.<ref>{{cite web |url=http://www.chipworks.com/en/technical-competitive-analysis/resources/recent-teardowns/2011/11/teardown-of-the-nikon-v1-camera-%E2%80%93-aptina-found/ |title=Chipworks: Teardown of the Nikon V1 camera – Aptina Found! |access-date=2019-12-13 |archive-url=https://web.archive.org/web/20120501142221/http://www.chipworks.com/en/technical-competitive-analysis/resources/recent-teardowns/2011/11/teardown-of-the-nikon-v1-camera-%E2%80%93-aptina-found |archive-date=2012-05-01 |url-status=dead }}</ref> The Nikon 1 series also includes an Epson graphic processor.

The Z-series, using Expeed 6 and 7 processors, also does not use additional co-processors.

==Alternative firmware== As with CHDK or Magic Lantern modified Canon digital cameras based on DIGIC processors, a group of programmers called "Nikon Hacker" develops custom firmware,<ref name=nhack/> making recent progress including an FR emulator for some DSLRs.<ref>{{Cite web |url=http://simeonpilgrim.com/blog/2011/11/07/how-to-decode-the-nikon-dslr-firmware/ |title=Simeon Pilgrim: How to decode the Nikon DSLR firmware |access-date=2012-02-29 |archive-date=2012-02-26 |archive-url=https://web.archive.org/web/20120226084844/http://simeonpilgrim.com/blog/2011/11/07/how-to-decode-the-nikon-dslr-firmware |url-status=live }}</ref><ref>{{Cite web |url=http://nikonhacker.com/index.php |title=Nikon Hacker: Board Index |access-date=2012-02-27 |archive-date=2012-02-26 |archive-url=https://web.archive.org/web/20120226023739/http://nikonhacker.com/index.php |url-status=live }}</ref><ref>{{Cite web |url=http://nikonrumors.com/2012/03/04/nikon-firmware-emulator-now-available.aspx/ |title=Nikon Rumors: Nikon firmware emulator now available |access-date=2012-03-05 |archive-date=2012-03-06 |archive-url=https://web.archive.org/web/20120306111134/http://nikonrumors.com/2012/03/04/nikon-firmware-emulator-now-available.aspx/ |url-status=live }}</ref> It was shown that Nikon uses the Softune integrated development environment together with an μITRON realtime kernel.<ref name=d70h/> Currently there is some modified firmware available mainly removing time based video and uncompressed NEF files restrictions,<ref name=nikhacsimeon >[http://simeonpilgrim.com/nikon-patch/Nikon-Patch.html Nikon patch] {{Webarchive|url=https://web.archive.org/web/20120531234134/http://simeonpilgrim.com/nikon-patch/Nikon-Patch.html |date=2012-05-31 }} simeonpilgrim.com</ref> but there is no stable alternative firmware available, as the project is still in an early state.

The Nikon supplied firmware-updates normally include the firmware A for the I/O processor and the firmware B to control ''Expeeds'' by integrated FR micro-controllers (different for the ARM-based ''Expeed 3'').

There are no alternate FW loads for Nikon Z-series cameras, only the multitude of menu options and frequent Nikon-provided FW updates.

===Project tasks=== Besides a general analysis of the hardware and software of the D7000, D5100 and D3100 and newer cameras (non-Z-series),<ref>[http://nikonhacker.com/wiki/Understanding_Firmware Understanding Firmware] {{Webarchive|url=https://web.archive.org/web/20130422011636/http://nikonhacker.com/wiki/Understanding_Firmware |date=2013-04-22 }} Nikonhacker Wiki</ref> the project focuses on: *Removing 5 minutes time video restrictions: Some modified firmware available<ref name=nikhacsimeon/> *High quality 1080p video with up to 64mbps *Uncompressed video (Clean HDMI Output) for the D5100 and others *Uncompressed NEF files<ref name=nikhacsimeon/> *WU-1a Wireless Mobile Adapter modifications to make it usable for other cameras *Analysis and decryption of Nikon battery communication *Analysis of Wireless Transmitter WT-3 and its communication *Removing third party battery blocking that Nikon introduced with some of the latest firmware versions *Development of an Nikon Emulator<ref>[http://nikonhacker.com/wiki/NikonEmulator NikonEmulator] {{Webarchive|url=https://web.archive.org/web/20121231163356/http://nikonhacker.com/wiki/NikonEmulator |date=2012-12-31 }} Nikonhacker Wiki</ref> The changelog<ref>[https://simeonpilgrim.com/nikon-patch/changelog.html Nikon Patch: Changelog] {{Webarchive|url=https://web.archive.org/web/20150519011658/http://simeonpilgrim.com/nikon-patch/ChangeLog.html|date=2015-05-19}} Simeon Pilgrim</ref> shows firmware also for newer variants like D800, D610, D5200 and D3200.

===DIAG raw hack=== This was not a firmware hack, but a method (Photopc – digital camera control<ref>{{Cite web |url=https://photopc.sourceforge.net/ |title=Digital Cameras on Fujitsu chipset |access-date=2012-02-28 |archive-date=2012-01-19 |archive-url=https://web.archive.org/web/20120119125804/http://photopc.sourceforge.net/ |url-status=live }}</ref>) calling an already implemented diagnostic mode on some old Nikon Coolpix cameras with Fujitsu Sparclite processors. Using this mode, they could write raw images.

==See also== * Canon DIGIC * Sony BIONZ * Sony Exmor

==References== {{Reflist|30em}}

Category:Nikon cameras Category:Camera firmware N Category:Parallel computing Category:SIMD computing Category:Image processors Category:Fujitsu microprocessors