FSM-IMX585 Datasheet

Sony IMX585AAQJ1-C Sensor Module

FRAMOS Sensor Module

FSM-IMX585 FSM-IMX585C v1.0d Datasheet
Specification
Model Name FSM-IMX585C (V1A)

Image Sensor
Sony
Vendor / Name
IMX585AAQJ1-C
Shutter Type CMOS Rolling Shutter
Technology / Grade Starvis2 / Security
Chromaticity Color
Optical Format 1/1.2“
Pixel Size 2.9 x 2.9 µm
Max. Resolution 8.3 Mpx / 3856 x 2180 px
Key Benefits & Features: 90.1 FPS (4-Lane)
Framerate (max.)
▪ 8.3 Mpx Sony CMOS Rolling Shutter sensor 30 FPS (2-Lane)
module, ready to embed! Bit Depth(s) 10 / 12 bit
▪ All FSMs are part of a rapid prototyping Interface
ecosystem, consisting of: Data Interface MIPI CSI-2 (2 / 4 Lane)
✓ Adapters to various processing boards Communication Interface I²C
✓ Design sources for deep embedding
Drive Frequency(s) 6 to 27 / 37.125 / 72 / 74.25 MHz
✓ Various accessories and design in services
Input Voltages 1.1V, 1.8V, 3.3V
Interface Connector Hirose DF40C-60DP-0.4V(51)
FSM-IMX585C (Color):
EEPROM (Sensor ID) Yes
Mechanical
Dimensions (HxWxD) 26.5 x 26.5
Environmental
-30°C to +85°C (function)
Operating Temperature
-10°C to +60°C (performance)
Storage Temperature -40°C to +85°C
Ambient Humidity 20% to 95% RH, non condensing
Software Support
V4L2 Based Device Driver
Driver Libargus / Argus Camera (ISP Tuned)
Isaac ROS compliant
NVIDIA Jetson Family:
Supported Platform(s)
AGX Xavier, Nano, TX2 (NX), Xavier NX
Software Version(s) NVIDIA JP4.6 / L4T32.6.1
Suggested Accessories & Adapters
Compatible FSA Type FSA-FT26/A
FSM-IMX585C/TXA_Devkit (TX2, AGX)
Recommended Devkit(s)
FSM-IMX585C/NVN_Devkit (Nano, NX)
Lens Mounts C/CS-Mount option
A matrix with compatible Sensor Adapters (FSA) and Processor Board Adapters
(FPA) for various setups can be found at the end of this document.

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

Mechanical Drawing

Sensor image optical center is in mechanical board center.

Connector Pinout
Type: Hirose DF40C-60DP-0.4V(51)
Mating Type: Hirose DF40HC(4.0)-60DS-0.4V(51)

D_DATA_0_N

D_DATA_2_N
D_DATA_0_P

D_DATA_2_P
SLAMODE0
XMASTER
Name

XCLR

INCK
GND
GND

GND

GND

GND

GND
SDA
XHS
3V3
3V3
1V8

XVS
SCL
NC
NC

NC

NC

NC
NC

NC

NC
NC

Pin 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59

Pin 1 according to print on PCB.

Pin 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60
D_DATA_3_N

D_DATA_1_N
D_DATA_3_P

D_DATA_1_P

D_CLK_0_N
SLAMODE1

D_CLK_0_P
TENABLE
Name

TOUT
GND
GND

GND

GND

GND

GND
1V8
1V8
1V1
1V1
NC

NC
NC
NC
NC
NC
NC
NC

NC

NC
NC

Signals are routed directly from image sensor to connector. Details on specific signals are described in the
respective image sensor datasheet.

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

Table of Contents
1 FRAMOS Sensor Module Ecosystem ............................................................................................................ 2
1.1 Ecosystem Overview............................................................................................................................. 3
1.2 Materials and Services .......................................................................................................................... 4
1.2.1 Native MIPI CSI-2 Modules ........................................................................................................... 5
1.2.2 Sub-LVDS, SLVS and SLVS-EC Modules ......................................................................................... 9
2 Software Package and Drivers .................................................................................................................... 10
2.1 Reference Software: NVIDIA Jetson Family ........................................................................................ 11
2.1.1 Platform and Sensor Device Drivers ........................................................................................... 11
2.1.2 Image Pre-Processing Examples ................................................................................................. 13
3 Ecosystem Compatibility Matrix ................................................................................................................. 15
3.1 Hardware Support .............................................................................................................................. 15

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

1 FRAMOS Sensor Module Ecosystem

The FSM Ecosystem consists of FRAMOS Sensor Modules, Adapters, Software and Sources, and provides one
coherent solution supporting the whole process of integrating image sensors into embedded vision products.

During the evaluation and proof-of-concept phase, off-the-shelf sensor modules with a versatile adapter
framework allow the connection of latest image sensor technology to open processing platforms, like the
NVIDIA Jetson Family or the 96boards.org standard. Reference drivers and sample applications deliver images
immediately after installation, supporting V4L2 and an optional derivate API providing comfortable integration.
Within the development phase, electrical design references and driver sources guide with a solid and proven
baseline to quickly port into individual system designs and extend scope, while decreasing risk and efforts.

To simplify and relieve the whole supply chain, all FRAMOS Sensor Modules and adapters are optimized and
ready for delivery in volume and customization with pre-configured lens holder, lens and further accessories.

Off-the-Shelf Hardware Kickstart Software Package

▪ FRAMOS Sensor Modules (FSM) from stock, ▪ Drivers with basic sensor integration:
ready for evaluation and optimized for initial ▪ V4L2 drivers for specific image sensors
mass production. ▪ Platform specific device tree overlays
▪ Versatile adapter framework, allowing ▪ Streamlined V4L2 library (LibSV) with
flexible testing of different modules, on comfortable and generic C/C++ API
different processing boards: ▪ Example applications demonstrating
▪ FRAMOS Sensor Adapter (FSA): initialization, configuration and image
Everything the specific sensor needs for acquisition
operation
▪ FRAMOS Processor Adapter (FPA):
Connect up to four FSM + FSA to a
specific processor board
▪ From lenses, mechanics and cables, all
needed imaging accessories from one hand

Further to the off-the-shelf hard- and software, the Ecosystem supports you on project basis with:

▪ Driver sources allowing the focus on application specific scope and sensor features
▪ Electrical references for FSA and FPA, supporting quick and optimized embedding of FSMs
▪ Engineering services via FRAMOS and its partners, allowing you to focus on your product’s unique value

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

1.1 Ecosystem Overview

The figure below shows a map of compatibility with all components inside the Ecosystem. Every element (or
hardware) and connection displayed in Green operates with native MIPI CSI-2 (D-PHY) data.

Sensor Module Sensor Module Adapter M S rocessing oard Adapter

rocessing oard
Su S, S S
S SE

M S etson T
etson A avier
M S M S
pass t roug

M S etson ano
etson T
n

n etson avier
onversion to
M S
S S S
M S www oards org
M S , ualco , iSilicon,
S, S S E Roc c ip, ilin and furt ers
pass t roug

S S E
S S E
S S, S arious evelop ent its
M S Series, ltraScale

Every component and connection displayed in Orange or Blue operates with proprietary (Sub-LVDS, SLVS) or
standardized (SLVS-EC) LVDS data, that requires further attention to the physical processing of the image data
by either data conversion or specific FPGA IP. Users of MIPI CSI-2 based processing systems are supported by
FSM specific data conversion located on dedicated FRAMOS Sensor Adapters (FSAs).

Lens Mount
(C/CS)

Sensor Module (FSM)

Sensor Adapter (FSA)
Flex Cable

Processor Adapter
(FPA)
Figure 1: Assembly of a typical Sensor Module Development Kit

Specification and compatibility of all individual components are listed in the appropriate chapter of the full
datasheet. Access to software and drivers is only granted with the purchase of the appropriate development
kit. Electrical design sources, support and services are provided on individual basis, they are not part of the
development kit or component purchase.
3

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

1.2 Materials and Services

Below you can find a list of materials and services as part of the FRAMOS Sensor Module Ecosystem.

Hardware
▪ FRAMOS Sensor Module Development Kits
▪ Individual Parts:
▪ FRAMOS Sensor Modules
▪ FRAMOS Sensor Adapters
▪ FRAMOS Processor Adapters
▪ FRAMOS Module Accessories (Cables, Mounts)

Software (part of the Development Kit)

▪ Software Package for NVIDIA Jetson AGX Xavier, Nano, TX2, TX2 NX and Xavier NX
▪ Software Package for DragonBoard 410c (96Boards)
▪ Xilinx FPGA reference implementation for SLVS-EC (Sony IMX421, IMX530)

Design Sources (on Project Basis)

▪ Software Driver Sources
▪ Electrical References for FSA, FPA (Schematics)

Design Services
▪ Off-the-shelf hardware customization including size, shape, connector and extended functionality
▪ Software customization and extension
▪ Additional processor board support
▪ Further sensor features and image (pre-)processing
▪ Integration of additional sensors
▪ Optimization for volume production
▪ Lens assembly and alignment
▪ System / solution development
▪ Production and integration

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 Version v1.0d from 2021-12-23
© FRAMOS 2021, information is subject to change without prior notice.
5
Datasheet

1.2.1 Native MIPI CSI-2 Modules

Global Shutters

Model Name FSM-IMX297 FSM-AR0144 FSM-IMX296 FSM-HDP230 FSM-IMX565

Shutter Type CMOS Global Shutter CMOS Global Shutter CMOS Global Shutter CMOS Global Shutter CMOS Global Shutter
Technology Pregius (Gen2)(Bin) - Pregius (Gen2) - Pregius S (Gen4)
Resolution [MP] 0.4 1 1.6 2.3 12.3
Resolution [HxV] 728 x 544 1280 x 800 1456 x 1088 1944 x 1204 4128 x 3008
59.9 FPS (4-Lane) 42.6 FPS (4-Lane)
Max. Framerate [FPS] 120.9 FPS (1-Lane) 60.3 FPS (2-Lane) 60.4 FPS (1-Lane)
59.9 FPS (2-Lane) 22.3 FPS (2-Lane)
Mono / Color Mono Color / Mono Mono Color / Mono Color / Mono
Sensor Manufacturer Sony ON Semiconductor Sony Pyxalis Sony
IMX297LLR / AR0144CSSM / IMX296LLR / HDPYX 230-G Mono IMX565AAMJ-C /
Sensor Name
IMX297LQR AR0144CSSC IMX296LQR / HDPYX 230-G RGB IMX565AAQJ-C
Application / Grade Sensing Industrial Sensing Automotive Industrial
Optical Format [inch] 1/2.9 1/4 1/2.9 1/2.5 1/1.1
Pixel Size [µm] 6.9 x 6.9 3x3 3.45 x 3.45 3.2 x 3.2 2.74 x 2.74
8 / 10 / 12 / 14 / 16
Pixel Bitdepth [bit] 10 bit 10 / 12 bit 10 bit 8 / 10 / 12 bit
bit

www.framos.com
Data Interface [Type] MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2
FSM-IMX585C

Data Interface [# Lanes] 1 1/2 1 2/4 2/4

Communication Interface I²C (4-wire serial) I²C I²C (4-wire serial) I²C I²C
37.125 / 54 / 74.25
Drive Frequency [MHz] 37.125 / 74.25 / 54 6 to 48 37.125 / 74.25 / 54 6 to 27
MHz
Input Voltages 1.2V, 1.8V, 3.3V 1.2V, 1.8V, 2.8V 1.2V, 1.8V, 3.3V 1.2V, 1.8V, 2.8V 1.1V, 1.8V, 2.9V, 3.3V
M12 or M12 or M12 or M12 or
Supported Lens Mounts C/CS-Mount option
C/CS-Mount options C/CS-Mount options C/CS-Mount options C/CS-Mount options
Board Dimensions [mm²] 26.5 mm x 26.5 mm 26.5 mm x 26.5 mm 26.5 mm x 26.5 mm 26.5 mm x 26.5 mm 26.5 mm x 26.5 mm
 Version v1.0d from 2021-12-23
© FRAMOS 2021, information is subject to change without prior notice.
Datasheet

Rolling Shutters (Part 1/3) – up to 5 MP

Model Name FSM-IMX327 FSM-IMX290 FSM-IMX462 FSM-IMX464 FSM-IMX335 FSM-AR0521

CMOS Rolling CMOS Rolling CMOS Rolling CMOS Rolling CMOS Rolling CMOS Rolling
Shutter Type
Shutter Shutter Shutter Shutter Shutter Shutter
Technology Starvis + NIR Starvis + NIR Starvis Starvis + NIR Starvis -
Resolution [MP] 2.1 2.1 2.1 4.2 5 5
Resolution [HxV] 1920 x 1080 1920 x 1080 1920 x 1080 2712 x 1538 2616 x 1964 2592 x 1944
60 FPS (4-Lane) 120 FPS (4-Lane) 120 FPS (4-Lane) 90 FPS (4-Lane) 60 FPS (4-Lane) 69 FPS (4-Lane)
Max. Framerate [FPS]
60 FPS (2-Lane) 60 FPS (2-Lane) 60 FPS (2-Lane) 30 FPS (2-Lane) 30 FPS (2-Lane) 34 FPS (2-Lane)
Mono / Color Color Color Mono Color / Mono Color / Mono Color / Mono
Sensor Manufacturer Sony Sony Sony Sony Sony ON Semiconductor
IMX290LLR / IMX335LLN / AR0521SR2M /
Sensor Name IMX327LQR1 IMX462LQR-C IMX464LQR-C
IMX290LQR IMX335LQN AR0521SR2C
Application / Grade Security Security Security Security Security
Optical Format [inch] 1/2.8 1/2.8 1/2.8 1/1.8 1/2.8 1/2.5
Pixel Size [µm] 2.9 x 2.9 2.9 x 2.9 2.9 x 2.9 2.9 x 2.9 2x2 2.2 x 2.2
Pixel Bitdepth [bit] 10 / 12 bit 10 / 12 bit 10 / 12 bit 10 / 12 bit 10 / 12 bit 8 / 10 / 12 bit
Data Interface [Type] MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2
Data Interface [# Lanes] 2/4 2/4 2/4 2/4 2/4 2/4

www.framos.com
Communication Interface I²C (4-wire serial) I²C I²C (4-wire serial) I²C I²C I²C
FSM-IMX585C

6 to 27 / 37.125 / 6 - 27 / 37.125 /
Drive Frequency [MHz] 37.125 / 74.25 37.125 / 74.25 37.125 / 74.25 10 to 48
74.25 74.25
Input Voltages 1.2V, 1.8V, 2.9V 1.2V, 1.8V, 2.9V 1.2V, 1.8V, 2.9V 1.2V, 1.8V, 2.9V 1.2V, 1.8V, 2.9V 1.2V, 1.8V, 2.7V
M12 or C/CS-Mount M12 or C/CS-Mount M12 or C/CS-Mount M12 or C/CS-Mount M12 or C/CS-Mount M12 or C/CS-Mount
Supported Lens Mounts
options options options options options options
Board Dimensions [mm²] 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5
 Version v1.0d from 2021-12-23
© FRAMOS 2021, information is subject to change without prior notice.
7
Datasheet

Rolling Shutters (Part 2/3) – 8 MP

Model Name FSM-IMX334 FSM-IMX485 FSM-IMX585 FSM-IMX678 FSM-IMX415 FSM-IMX715

CMOS Rolling CMOS Rolling CMOS Rolling CMOS Rolling CMOS Rolling CMOS Rolling
Shutter Type
Shutter Shutter Shutter Shutter Shutter Shutter
Technology Starvis Starvis + NIR Starvis2 Starvis2 Starvis + NIR Starvis + NIR
Resolution [MP] 8.3 8.3 8.3 8.3 8.4 8.4
Resolution [HxV] 3864 x 2180 3864 x 2180 3856 x 2180 3856 x 2180 3864 x 2192 3864 x 2192
72 FPS (4-Lane) 90.1 FPS (4-Lane) 72 FPS (4-Lane) 90 FPS (4-Lane) 90 FPS (4-Lane)
Max. Framerate [FPS] 60 FPS (4-Lane)
38 FPS (2-Lane) 30 FPS (2-Lane) 30 FPS (2-Lane) 44 FPS (2-Lane) 44 FPS (2-Lane)
Mono / Color Color / Mono Color Color Color / Mono Color Color
Sensor Manufacturer Sony Sony Sony Sony Sony Sony
IMX334LLR /
Sensor Name IMX485LQJ IMX585AAQJ1-C IMX678AAQR1 IMX415-AAQR IMX715AAQR1
IMX334LQR
Application / Grade Security Security Security Security Security Security
Optical Format [inch] 1/1.8 1/1.2 1/1.2 1/1.8 1/2.8 1/2.8
Pixel Size [µm] 2x2 2.9 x 2.9 2.9 x 2.9 2x2 1.45 x 1.45 1.45 x 1.45
Pixel Bitdepth [bit] 10 / 12 bit 10 / 12 bit 10 / 12 bit 10 / 12 bit 10 / 12 bit 10 / 12 bit
Data Interface [Type] MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2
Data Interface [# Lanes] 4 2/4 2/4 2/4 2/4 2/4

www.framos.com
Communication Interface I²C I²C I²C I²C I²C I²C
FSM-IMX585C

6 - 27 / 37.125 / 6 to 27 / 37.125 / 6 to 27 / 37.125 / 6 - 27 / 37.125 / 24 / 27 / 37.125 / 24 / 27 / 37.125 /

Drive Frequency [MHz]
74.25 74.25 72 / 74.25 74.25 72 / 74.25 72 / 74.25
Input Voltages 1.2V, 1.8V, 2.9V 1.2V, 1.8V, 2.9V 1.1V, 1.8V, 3.3V 1.1V, 1.8V, 3.3V 1.1V, 1.8V, 2.9V 1.1V, 1.8V, 2.9V
M12 or C/CS- M12 or C/CS- M12 or C/CS- M12 or C/CS-
Supported Lens Mounts C/CS-Mount option C/CS-Mount option
Mount options Mount options Mount options Mount options
Board Dimensions [mm²] 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5
 Version v1.0d from 2021-12-23
© FRAMOS 2021, information is subject to change without prior notice.
8
Rolling Shutters (Part 3/3) – equal or higher than 12 MP
Datasheet

Model Name FSM-IMX412 FSM-IMX577 FSM-IMX477 FSM-AR1335 FSM-IMX283

Shutter Type CMOS Rolling Shutter CMOS Rolling Shutter CMOS Rolling Shutter CMOS Rolling Shutter CMOS Rolling Shutter
Technology Starvis Starvis Starvis - Starvis
Resolution [MP] 12.3 12.3 12.3 13.1 20.2
Resolution [HxV] 4056 x 3040 4056 x 3040 4056 x 3040 4208 x 3120 5496 x 3694
59.9 FPS (4-Lane) 59.9 FPS (4-Lane) 59.9 FPS (4-Lane) 27.2 FPS (4-Lane)
Max. Framerate [FPS] 24.7 FPS (4-Lane)
30 FPS (2-Lane) 30 FPS (2-Lane) 30 FPS (2-Lane) 13 FPS (2-Lane)
Mono / Color Color Color Color Color Color
Sensor Manufacturer Sony Sony Sony ON Semiconductor Sony
AR1335CSSM /
Sensor Name IMX412-AACK IMX477-AAPK IMX577-AACK IMX283CQJ
AR1335CSSC
Application / Grade Security Security Security Industrial Audio/Video
Optical Format [inch] 1/2.3 1/2.3 1/2.3 1/3.2 1
Pixel Size [µm] 1.55 x 1.55 1.55 x 1.55 1.55 x 1.55 1.1 x 1.1 2.4 x 2.4
Pixel Bitdepth [bit] 10 / 12 bit 8 / 10 / 12 bit 8 / 10 / 12 bit 8 / 10 bit 10 / 12 bit
Data Interface [Type] MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2
Data Interface [# Lanes] 2/4 2/4 2/4 2/4 4
Communication Interface I²C (CCI) I²C (CCI) I²C (CCI) I²C I²C
Drive Frequency [MHz] 6 / 12 / 18 / 27 6 to 27 6 to 27 6 to 48 6 to 27
Input Voltages 1.05V, 1.8V, 2.75V 1.05V, 1.8V, 2.8V 1.05V, 1.8V, 2.8V 1.2V, 1.8V, 2.7V 1.2V, 1.8V, 2.9V
M12 or M12 or M12 or M12 or
Supported Lens Mounts C/CS-Mount option
C/CS-Mount options C/CS-Mount options C/CS-Mount options C/CS-Mount options
Board Dimensions [mm²] 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5 26.5 x 26.5

www.framos.com
FSM-IMX585C
 Version v1.0d from 2021-12-23
© FRAMOS 2021, information is subject to change without prior notice.
9
Datasheet

1.2.2 Sub-LVDS, SLVS and SLVS-EC Modules

Model Name FSM-IMX264 FSM-IMX304 FSM-IMX530

Shutter Type CMOS Global Shutter CMOS Global Shutter CMOS Global Shutter
Technology Pregius (Gen2) Pregius (Gen2) Pregius S (Gen4)
Resolution [MP] 5.1 12.4 24.5
Resolution [HxV] 2464 x 2056 4112 x 3008 5328 x 4608
SLVS-EC: 106.9 FPS (8-Lane)
Max. Framerate [FPS] CSI-2: 35.7 FPS (4-Lane) CSI-2: 23.4 FPS (4-Lane)
CSI-2: 30 FPS (4-Lane)
Mono / Color Color / Mono Color / Mono Color / Mono
Sensor Manufacturer Sony Sony Sony
Sensor Name IMX264LLR / IMX264LQR IMX304LLR / IMX304LQR IMX530-AAMJ / IMX530-AAQJ
Application / Grade Industrial Industrial Industrial
Optical Format [inch] 2/3 1.1 1.2
Pixel Size [µm] 3.45 x 3.45 3.45 x 3.45 2.74 x 2.74
Pixel Bitdepth [bit] 12 bit 12 bit 8 / 10 / 12 bit
Data Interface [Type] SubLVDS SubLVDS SLVS, SLVS-EC
Data Interface [# Lanes] 4 4/8 1/2/4/8

www.framos.com
Communication Interface I²C (4-wire serial) I²C (4-wire serial) I²C (4-wire serial)
FSM-IMX585C

Drive Frequency [MHz] 37.125 / 54 / 74.25 37.125 / 54 / 74.25 37.125 / 54 / 74.25

Input Voltages 1.2V, 1.8V, 3.3V 1.2V, 1.8V, 3.3V 1.1V, 1.8V, 2.9V, 3.3V
Supported Lens Mounts C/CS-Mount option C/CS-Mount option C/CS-Mount option
Board Dimensions [mm²] 28 x 28 28 x 28 28 x 28
 FSM-IMX585C Datasheet

2 Software Package and Drivers

As FRAMOS we know that the getting started with a new technology is the biggest challenge. The idea behind
the Software Package is to enable embedded software engineers to get quickly to a streaming system and
provide at the same time all tools that are needed to extend and adapt it according the individual needs of the
application.

What the software package and driver are:

▪ A reference for a custom sensor implementation
▪ Demonstrating how to use the required interfaces
▪ Demonstrating how to communicate with the image sensor
▪ Demonstrating how to generaly initialize and configure the image sensor
▪ Provide initial image streaming output to the user space
▪ Demonstrating how to run basic image processing on pixel data

Supported Processor Platforms

The table below shows which platforms are supported by the standard driver package, and how many FSMs
can at maximum be operated in parallel.

NVIDIA
96Boards Xilinx
NVIDIA NVIDIA Jetson Nano, DragonBoard
Sensor Module Consumer Development
Jetson TX2 AGX Xavier TX2 NX, Xavier 410c
Edition Boards
NX
FSM-AR0144 4 2
FSM-AR0521 4 2 2
FSM-AR1335 4 2
FSM-HDP230 4 4 2
FSM-IMX264 2 4 -
FSM-IMX283 2 4 -
FSM-IMX290 4 2 2
FSM-IMX296 4 2 2
FSM-IMX297 4 2
FSM-IMX304 2 4 -
FSM-IMX327 4 2 2
FSM-IMX334 2 4 - HW only, driver development
FSM-IMX335 4 2 on project basis.
FSM-IMX412 4 2 2
FSM-IMX415 4 2
FSM-IMX462 4 2
FSM-IMX464 4 2
FSM-IMX477 4 2
FSM-IMX485 4 2
FSM-IMX565 4 2
FSM-IMX577 4 2
FSM-IMX585 4 2
FSM-IMX678 4 2
FSM-IMX715 4 2
FSM-IMX530 2 4 - 11

Table 1: Ecosystem Software Package - Supported number of FSMs per processing board

1
SLVS-EC based FPGA reference implementation as part of the SLVS-EC RX IP Core offering. 1

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

2.1 Reference Software: NVIDIA Jetson Family

The software package provided with the Development Kits of the FRAMOS Sensor Module Ecosystem provided
for NVIDIA Jetson platforms provides a reference implementation of sensor and device drivers for MIPI CSI-2.
It contains a minimum feature set demonstrating how to utilize the platform specific data interface and
communication implementation, as well as the initialization of the image sensor and implementation of basic
features.

Package Content:
▪ Platform and device drivers with Linux for Tegra Support
▪ V4L2 based subdevice drivers (low-level C API)
▪ Streamlined V4L2 library (LibSV) providing generic C/C++ API
▪ Display Examples:
▪ OpenCV (Software)
▪ LibArgus (Hardware)

Supported Devices:
▪ NVIDIA Jetson Nano Developer Kit (B01)
▪ NVIDIA Jetson TX2 Developer Kit
▪ NVIDIA Jetson TX2 NX Developer Kit
▪ NVIDIA Jetson Xavier NX Developer Kit
▪ NVIDIA Jetson AGX Xavier Developer Kit

2.1.1 Platform and Sensor Device Drivers

The driver divides into two main parts that are configured in separate ways – the Image Modes and the General
Features of the image sensor.

Image Modes
These are major attributes that have impact to the image data stream formatting. They require a static pre-
configuration within the device tree (DT):

▪ Image / streaming resolution

▪ Pixel format / bit depth
▪ Data rate / lane configuration

Each driver provides access to 3 – 5 pre-built configurations, reflecting the main operation modes of the
imager. Beside the full resolution, that is always available, they allow to receive image streams in common
video resolutions like VGA, Full HD and UHD as they are supported or make sense by the imagers, and utilize
sensor features like ROI and binning.

They act as an example for implementation and usage and are available as source. Due to the size limitation
of the device tree, it is not possible to integrate an extensive set of options.

11

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

General Features
These are attributes of the image sensor that do not manipulate the data stream formatting. The drivers
provided with the Software Pack integrate the sensor features as shown in the table below.

Synchronizing Master
IS Mode (Master / Slave)

Test Pattern Output

Gain (Analog / Digital)

Sensor Mode ID
Pre-Implemented
Exposure Time
Flip / Mirror

HDR Output
Features per
Frame Rate

Black Level

Broadcast
Data Rate
Model

FSM-AR0144
FSM-AR0521
FSM-AR1335
FSM-HDP230
FSM-IMX264
FSM-IMX283
FSM-IMX290
FSM-IMX296
FSM-IMX297
FSM-IMX304
FSM-IMX327
FSM-IMX334
FSM-IMX335
FSM-IMX412
FSM-IMX415
FSM-IMX462 V4L (libsv) and libargus
FSM-IMX464 V4L (libsv)
FSM-IMX477 Not Supported/Implemented
FSM-IMX485
FSM-IMX530
FSM-IMX565
FSM-IMX577
FSM-IMX565
FSM-IMX585
FSM-IMX678
FSM-IMX715
Table 2: Supported sensor features on NVIDIA Jetson Family

Further features, as they are supported by the image sensor, can be integrated into the driver sources using
the image sensor datasheet.

12

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

2.1.2 Image Pre-Processing Examples

The provided image processing examples show the general mechanisms of data handling for an image
processing using 3rd-party libraries. The OpenCV example provides data that is raw (mono) or demosaiced
(color) and not further optimized for visual experience, while the LibArgus examples leverages the discrete ISP
(Image Signal Processor) inside the Jetson SoC to optimize image reproduction.

Argus Camera Example:

▪ Using hard ISP in NVIDIA Jetson SoCs, most performant option for image preprocessing
▪ Only applicable for color sensors color processing can’t e disa led
▪ Most performant option
▪ Utilizing libArgus closed source library, support and tuning on individual basis through FRAMOS
▪ Example Implementation: Shows Demo Tuning per FSM Devkit

Xavier Tegra X2 Tegra X1

(AGX, NX) (TX2, TX2 NX) (TX1, Nano)
Performance
Max. # of streams through ISP 16 12 6
Pixel Bandwidth (max.) 2 Gpix/s 1.4 Gpix/s 1.4 Gpix/s
Image Resolution (max.) 64 MP 24 MP 24 MP
Image Width (max.) 6144 px 6144 px 6144 px
Table 3: ISP capabilities / limitations of NVIDIA Jetson Family

The software package provided with our FSM Devkits contains a functionally and performance limited example
configuration for the Jetson ISP. The configuration is sensor and lens related and demonstrates the
combination of our standard kit in environments that are illuminated with fluorescent light, like in office or
laboratory.

Supported Features in Default Configuration

Lens Considered (Type) Yes (Devkit Lens)
IR Cut Filter (Type) Yes (650nm/50%)
Sensor Configuration Driver Default
Demosaic Yes2
Black Level Compensation Yes (Calibrated)
Bad Pixel Correction Yes (Calibrated)
Color Correction Yes (Calibrated)
Auto White Balance (A, TL84, D65) Limited (Calibrated for TL84 only)
Manual White Balancing Limited (Not Calibrated)
Lens Shading / Falloff Correction Limited (Calibrated for Devkit lens)
Noise Reduction Limited (Not Calibrated)
Sharpening Limited (Not Calibrated)
Auto Exposure, Gain, Gamma,
Color/Tone, Contrast Tuning
Requires Application Specific Tuning
Table 4: Default tuning of NVIDIA Jetson, supplied with FSM Devkits

2
e osaicing is always active and can’t e disa led For onoc ro e sensors refer to li S to ypass t e S 13

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

To archive best performance and stable results also in variable lighting conditions, a fully featured calibration
will be required. As NVIDIA camera partner, FRAMOS provides full ISP configurations for standard setups on
request. Further, full custom calibration services considering lens and application specific requirements for
sophisticated applications are provided on per project basis.

OpenCV Example:
▪ Open software library
▪ Easy to use and large feature set
▪ Very resource hungry (CPU)
▪ Not recommended for pre-processing
▪ Example Implementation: Demosaicing, Displaying

Due to limited performance and extreme resource utilization, the image processing support utilizing the CPU
will not be further enhanced.

14

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

3 Ecosystem Compatibility Matrix

3.1 Hardware Support

The following matrix shows the compatibility of FSMs, FSAs and FPAs to each other. The FSAs differentiate to
each other by supplied voltages, power up sequence, generated clock (oscillator) and physical attributes.

Sensor Modules with MIPI CSI-2 (D-PHY) Output

FSM-IMX290
FSM-IMX327
FSM-IMX334
FSM-IMX335
FSM-IMX412 FSM-IMX462 FSM-IMX565
FSM-IMX477 FSM-IMX464 FSM-IMX296 FSM-AR0521 FSM-IMX415 FSM-IMX585
Item FSM-IMX577 FSM-IMX485 FSM-IMX297 FSM-AR1335 FSM-IMX715 FSM-IMX283 FSM-AR0144 FSM-HDP230 FSM-IMX678
FPA-4.A/TXA
FPA-A/NVN
FSA-FT1/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN4
FSA-FT3/A
FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN
FSA-FT6/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN
FSA-FT7/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN
FSA-FT11/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FSA-FT12/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN
FSA-FT13/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN
FSA-FT19/A FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FPA-A/NVN
FSA-FT26/A
FPA-2.A/96B
FPA-ABC/XX13
Table 5: Ecosystem Compatibility Matrix – Native CSI-2 (D-PHY) FSMs

3
Not verified, Xilinx Development Board with hard MIPI CSI-2 / D-PHY interface.
4
FSM-IMX334 is not supported due to the sensor requiring 4-lanes MIPI. 15

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.
 FSM-IMX585C Datasheet

Sensor Modules with (Sub-) LVDS and SLVS Output

Item Data Output (FSA) FSM-IMX264 FSM-IMX304 FSM-IMX421 FSM-IMX530
FPA-4.A/TXA
FSA-FT14/A-00G MIPI CSI-2 FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FSA-FT15/A-00G MIPI CSI-2 FPA-2.A/96B
FPA-ABC/XX13
FPA-4.A/TXA
FSA-FT18/A-00G MIPI CSI-2 FPA-2.A/96B
FPA-ABC/XX13
FPA-ABC/XX1
FSA-FT18/BC SLVS, SLVS-EC
FPA-C/AGX

FSA-FT20/BC SLVS-EC FPA-ABC/XX1

Table 6: Ecosystem Compatibility Matrix – Sub-LVDS, SLVS and SLVS-EC FSMs

Version v1.0d from 2021-12-23

www.framos.com © FRAMOS 2021, information is subject to change without prior notice.