ARRI D-21 VS SONY FS700

Team Sony NEX-FS700
Carly Cerquone – Compositing || Head of Production
David Villarreal – Editing || Cinematography

Team Arri D-21
Jordan Westhoff – DIT || Image Processing
Matt Bernstein – Editing || Cinematography

Each camera was judged on its sharpness, high fidelity color reproduction, quality of MTF, scene dynamic range, noise presence, ease of compositing, overall efficiency, and a multitude of other attributes. As was required, a shootout video was completed and can be watched below. In addition to the video, our group was also required to complete a formal analysis of all our findings. That too can be viewed below. If you don’t want to read 53 pages worth of our project, I have written a more concise version here.

Sony NEX-FS700

Super 35mm Exmor CMOS sensor
Native aspect ration of 16:9
Active Imaging area of 32.6mm x 13.3
Standard EF lens mount
Electronic viewfinder
Capable of outputting 4K RAW data streams to an external recorder via a 3G HD-SDI connection
Supports a variety of HD video formats that can be recorded onto SD flashcards without the use of additional hookups

ARRIFLEX D-21

Standard super 35mm CMOS sensor
Native aspect ratio of 4:3
Active resolution of 2880 x 2160 pixels
Allowable aspect ration of 16:9
Active cropped resolution of 2880 x 1620 pixels
Commonly paired with anamorphic lens setups
Standard PL lens mount
Equipped with a rotating mechanical shutter array
Features a fully optical eyepiece
Offers a variety of recording formats, including ARRI’s ARRI-RAW format
Capable of achieving 12-bit raw image data by connection to a recorder via dual-link HD-SDI connection, each of which is capable of transmitting 1920 x 720 pixels of information

FIELDTESTING 

Over the course of three shooting days, we conducted eight different tests to analyze the performance of each camera with regard to:

Scene Dynamic Range
Scene Noise
Ease of Compositing
Chroma Key and Skin Tones
ISO 12233 Chart || Aliasing and MTF
OECF || Dynamic Range
Macbeth Colorchecker Chart || Color Reproduction
Kodak Grey Card || Noise Anaylsis
Skin Tone Reproduction

Dynamic Range Latitude

Through a carefully crafted lighting setup, and the use of our studio’s wild walls we were able to create a scene spanning well over 12 stops of exposure latitude. Scene highlights read +9 stops over normal, and scene shadows reached a maximum of -6.1 stops below normal. Many stationary objects of varying specular attributes were placed within the scene, and one actress was cast to move about the set-up, demonstrating the lighting changes on skin tones, while another was placed under the table on the left to test the camera’s capability to reproduce shadow detail.Both the Sony NEX-FS700 and the ARRIFLEX D-21 were outfitted with Zeiss CP2 35mm lenses set to an f-stop of 5.6 using a relative ISO 200
Despite the noisiness in the rendered shadows, the ARRIFLEX D-21 produced a more expansive visual dynamic range than that of the FS-700. It was capable of rendering developed mid tone information, and was able to deliver image detail in high exposure regions that the FS-700 was not. That being said, the FS-700 out-performed the D-21 in the shadow regions, delivering significantly more image detail.

SCENE NOISE

Noise is, of course, an unavoidable product of imaging systems. It can be caused by shutter speed, gain, and well size, and many other parts of the imaging process.
To measure the scene noise generated by the electronic sensors of both of these cameras, a scene was constructed consisting of multiple exposure zones, memory colors, various stationary objects, and a centerpiece grey card. The scene was shot by each camera at normal exposure, two stops above normal, and two stops below normal.
The scene was analyzed visually as well as computationally.The FS700 was far less noisy than the D-21 in all scenarios.

Compositing Ease

To test each camera’s ability to produce a key-able image, we set up a compositing scene that included soft edges, high frequency patterns, partial overlap, frenzied motion, and a mid-range frequencies. Because the ease in which a scene can be composited is largely under the discretion of the compositor, and the tools available to him/her, I gave myself only the basic in-school technologies, and half an hour to work with.I chose to work with Adobe After Effects CS6, as that is what I am most familiar with. As expected, the areas of high frequency resulted in considerable artifacting, and in areas of movement, the key fails to discern between the object and the background due to its thickness, motion blur, and reflectivity. All in all, though, the key was straightforward and simple.
The D-21 was slightly more difficult and time consuming to pull an acceptable key from than the FS-700. That being said, the resulting image produced by the composited D-21 footage was more believable in the long run. The D-21 was able to preserve more detail than the FS-700, providing a stronger composite.

System MTF

By imaging a standardized ISO 12233 chart, we were able to test visible aliasing and MTF reproduction abilities of each camera. Though the chart does provide a large amount of information, we focused our analysis on the areas providing the vertical and horizontal MTF slant edge tests, and the delta frequency patterns of various frequencies. They can be seen outlined in red. 
Modulation Transfer Function (MTF) is defined as a methodology to analyze the ability of a system to capture frequency information. it is the spatial frequency response of an imaging system or the functional delta between a standardized frequency and a lower/higher frequency from the nyquist. It can also be defined as “the ratio of contrast in an image to the contrast of the object in the real scene (in linear photometric terms) as determined at various frequencies and negating non0unity transfer characteristics”.
Each independent physical element of an imaging system possesses its own independent MTF, an must be added to a multiplicative sum to determine the MTF of the entire system.
The ISO 12233 chart provides aliasing information as well, and can be observed in the area labeled “3".

The calculated MTF of each system is very similar, though the D-21 does manage to maintain a slightly better slope reading, especially on the X-axis after the 0.4 marker. Both cameras used the same 50mm Zeiss Compact Prime lens, so any MTF failures as a result are carried across both systems.

As for aliasing, the signal degradation occurs sooner for the FS700 than the D-21. The FS-700 begins to loose its resolve at the 5 mark, while the D-21 maintains power until the 7.

Dynamic Range

The Opto-Electric Conversion Function (OECF) of each camera is the process by which it senses a luminance input from a scene, and converts the information into a displayable image.  To accurately determine the dynamic range of each camera, an OECF chart was properly imaged at a normal, +2 and -2 exposure, and analyzed computationally. The FS-700 renders approximately 10 stops of dynamic range, and the D-21 captures just over 11. The D-21 renders midtones and highlights significantly better than the FS-700, though failed to render shadows without great amounts of noise.

Color Reproduction

To determine color reproduction, the MacBeth Color Checker chart was imaged. The D-21 provides a more true color performance than the FS-700, though render the yellows more desaturated than the reds and greens. The FS-700 displays a more yellow bias over all, and displays noticeable saturation in the true yellow patches.

Grey Card Noise

To determine the noise inherently present in each camera, the standard deviation of values from normal in each channel was measured, squared to remove uncertainty and generate a value for the entire grey card, and analyzed. The FS-700 shows considerably less noise across the entire range of exposure values than the D-21. This only stands to back up our previous observations.

Results

The Arri D-21 has an advantage in dynamic range, specifically in the highlight detail, color reproduction, and resolving power, where as the FS700 maintains an advantage in shadow detail. It is important to take into consideration that the ARRIRAW footage was scrutinized mainly against the FS700 HD footage, and not against it's its 4K RAW. Had that comparison been made, the FS700 would have surpassed the ARRI D-21 in the departments of aliasing, noise, and skin tone reproduction.