2018 3rd IEEE International Conference on Image, Vision and Computing

Integrated Design of Software for CCD Imaging Circuit Based on FPGA

Zhen Mingyue 1, Su Jian2, Ge Jianyun1

1
DFH Satellite Co., Ltd., Beijing, China
2
Beijing Institute of Space Mechanics & Electricity, Beijing, China
e-mail: ywmjz@163.com, sujian1003@163.com

Abstract—In order to meet the needs for the integration and timing circuit is to provide clock, synchronization signal for
miniaturization of CCD imaging circuit, a software integrated signal processing circuit, and send instruction information to
design method is provided in this paper. Using FPGA as the signal processing circuit. The system diagram of the CCD
core of the imaging circuit, the three parts of the traditional imaging circuit realized by traditional design method is
CCD imaging circuit, which is focal plane software, signal shown in Fig. 1.
processing software and integration timing software, are In the traditional design method, the three parts of the
integrated into a piece of FPGA, mainly generating the timing imaging circuit use FPGA as their respective main control
drive signal needed for CCD imaging, configuring the A/D chips, which not only makes the integration of the hardware
converter to realize analog to digital conversion, coding and
and software low, but also increases the development cost of
synthesizing the converted image data and arranging the data
transmission format, realizing the communication between the
the whole circuit. At the same time, due to the derating
imaging circuit and the outside by the telemetry and requirements of the FPGA design, it also causes a certain
telecontrol three wire interface, so as to receive the auxiliary degree of waste of logical resources [1]. In order to improve
data on the satellite and the adjustment instructions of the the integration of imaging circuit to meet the need of
imaging parameters. The experimental results show that the miniaturization of the camera, a new software architecture
integrated software design of this paper can realize all for the CCD imaging circuit is researched and designed in
functions of focal plane software, signal processing software this paper. On the basis of the traditional imaging circuit,
and integration timing software. The imaging effect is clear, only the FPGA of signal processing circuit is retained. The
the hardware circuit is simplified and the software integration function of FPGA software of focal plane circuit and
is improved, and it has high engineering application value. integration timing circuit is integrated into the FPGA of
signal processing circuit to realize the integrated design of
Keywords-CCD; software integration; imaging circuit; the software so as to achieve the goal of improving the
FPGA integration and saving hardware and software resources.
optical focal plane circuit signal processing circuit
I. INTRODUCTION signal
Analog signal

CCD FPGA Data

CCD is a kind of charge coupled device image sensor. It Analog
Correlated signal
Digital
signal
output
A/D
uses the principle of photoelectric conversion to convert the Management integration timing
Clock,
synchronization
Double
Sampling
convertion
FPGA

image information of optical signal into electrical signal, controller

signal
circuit n
and instruction

which is widely used in the image acquisition system. In the FPGA

imaging circuit of the satellite camera, CCD receives the

optical signal from the optical system to achieve Figure 1. Block diagram of traditional CCD imaging circuit system
photoelectric conversion. The converted electrical signal is
converted into digital signal after the Correlated Double
II. THE COMPOSITION OF INTEGRATED IMAGING CIRCUIT
Sampling and A/D conversion operation. The digital signal is
transmitted to the digital signal processor to complete the In the integrated design, the three parts of the software
processing and imaging of the image signal. are implemented in a single FPGA. The block diagram of the
The traditional CCD imaging circuit mainly includes integrated CCD imaging circuit is shown in Fig. 2. The
three parts: focal plane circuit, signal processing circuit and imaging circuit consists of focal plane circuit and signal
integration timing circuit. The three parts of the circuit have processing circuit. The focal plane circuit includes the CCD
their own FPGA as the core processor to realize the driving circuit and the analog signal filter amplifying circuit.
corresponding function. The function of FPGA software of The timing driving signal of CCD is provided by the FPGA
focal plane circuit is to provide the timing driving signal with of signal processing circuit; the signal processing circuit
a strict phase relationship to ensure the normal operation of includes A/D converter, FPGA, PROM, SRAM, the data
CCD. The function of FPGA software of signal processing transmission interface and the telemetry and telecontrol
circuit is to drive the A/D converter to complete analog-to- interface. Compared with the traditional design, the
digital conversion, convert analog signal output from CCD to integrated design of the imaging circuit only retains the
digital signal, synthesize digital signal and convert into a FPGA of signal processing circuit as the core processor,
data transmission format, then output to a data transmission reducing the number of FPGA, but the design complexity of
subsystem. The function of FPGA software of integration

978-1-5386-4991-6/18/$31.00 ©2018 IEEE 624

the FPGA after integrated design is increased compared with 1) Receiving and interpreting the three-wire instructions,
the traditional design [2]. outputting the interpreted instructions to the corresponding
The core controller of signal processing circuit adopts V2 module for response;
series FPGA of Xilinx company, and the chip logic resources 2) Generating the configuration driving signal of the
are sufficient to meet the design needs of the current system.
A/D converter, driving the A/D converter to perform
The analog to digital conversion uses a dual-channel A/D
converter of Analog Devices company, which can perform analog-to-digital conversion on the analog signal, receiving
correlated double sampling on CCD analog signal with a the converted image data, synthesizing and transmitting the
quantization accuracy of 14 bits, and the gain, clamping and data according to the corresponding format;
other parameters of the device can be adjusted by changing 3) Producing the horizontal driving timing signal and
the configuration of internal registers. The working process vertical driving timing signal required for CCD, driving the
of the CCD imaging circuit is as follows: the optical signal is CCD to work normally, and providing the correct pixel
projected onto the photosensitive area of CCD through the signal for A/D converter.
optical lens, and FPGA acts as a core controller to generate TOP
the timing driving signal required for the CCD operation, the
optical signal is converted into the cumulative charge and the
output is finally produced in the form of voltage. The analog configuring Telemetry
Global signal
A/D and
signal output by the CCD is converted into digital signal management
converter telecontrol
after correlated double sampling and analog-to-digital
Generating timing Data
conversion. FPGA receives the A/D quantized digital signal signal of CCD processing
for encoding and data synthesis, and converts it into the
specified data format for transmitting. The signal processing Figure 3. FPGA logical function partition
circuit receives telecontrol instructions such as the auxiliary
data and the imaging parameters sent by the management Power-on/reset

controller through the three wire serial port [3]. FPGA Reset delay
adjusts the working timing of CCD and the working state of
the A/D converter according to the received instructions. Internal clock and
synchronization
generating
focal plane circuit Analog
A signal processing circuit
signal
Filter and A/D PROM
Driving A/D
optical amplification Convertion Data
t
Digital Three wire Horizontal Vertical
signal data output
CCD siignal instruction driving signal driving signal
FPGA transmission
CCD driving receiving Data receiving generating generating
interface
signal
Driving
device SRAM Instruction
Data synthesizing
deecoding
Driving signal
Instruction Data transmitting output
Telemetry and
output
telecontrol interface

Figure 2. Block diagram of imaging circuit system after integrated design

End

Figure 4. Workflow diagram of FPGA software

III. IMPLEMENTATION OF SOFTWARE DESIGN
The design core of the CCD imaging system is the logic A. The CCD Timing Generation Module
function design of the FPGA. The hierarchical structure
diagram of the FPGA software divided by function is shown The main function of the timing generation module is to
in Fig. 3. The logic function of FPGA software includes generate various driving signal required for the CCD to
generating the global management signal, generating the operate normally, and adjust the timing signal for driving the
timing driving signal of the CCD, generating the CCD according to the control instructions, so that the CCD
configuration timing of the A/D converter, synthesizing the operates in different working modes. The conventional
image data and converting the data format, receiving the driving signal for CCD in this paper includes vertical transfer
telecontrol instructions and returning the telemetry data[4]. pulses I and S, in which I is the frame transfer pulse, S is the
The integrated FPGA software working flowchart is line transfer pulse, the horizontal transfer pulse R, the control
shown in Fig. 4. After power-on or reset, the reset signal is signal DG of the clear inverted charge and the reset signal
delayed to ensure that the reset signal is stable and effective, ΦR, the relationship between the timing signal of the whole
and then the global signal management module generates frame mode output is shown in Fig. 5. RØHV is a high
various clock signal and synchronization signal required for voltage sine wave signal that is generated based on the line
internal working. After the clock signal and synchronization synchronization signal after the system power reset is
signal is generated, the software workflow is divided into completed.
three main processes according to the function of integration T1 is the line clearing stage. At this time, the vertical
timing software, signal processing software, and focal plane transfer signal I and S output, and the purpose is to remove
software: the charge in the photosensitive area before the integration
stage comes. T2 is the integration stage. At this time, the

625
vertical transfer signal and horizontal transfer signal are all position, H signal assisted testing is required. The position of
fixed level, which is the time interval after the charge is the rising edge of the H1 signal is equal to the position of the
poured out to the vertical transfer stage. T3 is the charge SHD sampling pulse, and the position of the falling edge of
transfer and readout stage, in which the vertical transfer the H1 signal is equal to the position of the SHP sampling
signal I and S continue to output until a certain line is output, pulse. The signal position of the SHP and SHD sampling
then maintain a fixed level. T4 is the line transfer stage. In pulses can be determined by measuring and changing the
this stage, the vertical transfer signal I maintains a fixed level, phase relationship of the H1 signal corresponding to the
the vertical transfer signal S is output according to the line CCD signal through an oscilloscope.
synchronization period, the horizontal transfer signal R is The normal work of AD is to complete the change of
continuously output. The RØHV signal is always valid with function on the basis of correctly configuring the internal
the horizontal transfer signal R. Timing requirements for register. The internal register is configured by FPGA through
CCD reset pulse signal is shown in Fig. 6. the three wire serial interface. The width of the register is 32
IΦ1/IΦ2
bits, which is composed of 8 bits address and 24 bits data.
IΦ3/IΦ4 The falling edge of serial clock is aligned with the jumping
SΦ1/SΦ2 edge of data. The timing relationship of the configuration is
SΦ3/SΦ4

Rž1
shown in Fig. 7. After completing the initial configuration of
Rž2 the A/D converter, it is also necessary to send timing signal
Rž3 such as pixel clock, line synchronization, and frame
žR

RžHV
synchronization to the A/D converter.
DG
T1 T2 T3 T4

Fsyn

Frame period

Figure 5. Transfer Timing diagram of CCD

Figure 7. Configuring timing of A/D converter

C. Image Data Processing Module

The image data format output by the CCD include over
scan pixels, dark pixels, and effective pixels. The data
processing module mainly encodes and synthesizes the
Figure 6. Timing requirements for CCD reset pulse signal parallelized digital signal after AD sampling, and removes
invalid pixels before and after the valid data. Only valid
B. The AD Converter Configuration Module pixels are reserved for data synthesis, and the synthesized
Reset noise is the main noise of CCD during the working image data is converted into the format of the data
process. In order to eliminate the influence of reset noise on transmission interface for output through dark pixel
CCD imaging and improve image quality and signal-to-noise correction and inconsistency correction. In order to output
ratio, the correlated double sampling (CDS) technique is the valid data accurately, the number and position of the over
needed[5]. In this paper, the analog signal is converted into a scan pixels and the dark pixels must be determined so as to
digital image signal by double sampling of the A/D converter. generate respective enable frame, and the phase relationship
The reset noise accompanied by the reference level and the of them is shown in Fig. 8.
effective level of the analog signal is unevenly distributed in
SamClk
one pixel cycle. The device is equipped with 2 pulses, the
clamp pulse corresponds to one register, denoted by SHP, Reset
and the sampling pulse corresponds to another register, Syn
represented by SHD. The sampling pulse continuously DarkFEn
T1

samples the reference level and effective level of the same DataEn
TT2

analog level, and the location is distributed in a flat stepped DarkBEn

area as much as possible. The calculated CCD voltage is the TT3

output CDS value, that is, the difference between the Figure 8. The phase relationship of data processing module
reference level and the effective level, so that the noise
source can be eliminated. The register is configured through T1 is the time for the first pixel to the rising edge of the
a three wire serial port to adjust the position of the two line synchronization. Taking into account the delay caused
sampling pulses. by analog signal transmission and inter-board transmission,
Since the SHP and SHD signal is generated internally in T1 is instantiated into the internal module and the
the device and cannot be directly observed for correct corresponding parameter value is generated by the measured

626
value of the oscilloscope. DarkFEn is the enabling of over Use an oscilloscope to observe the vertical transfer signal
scan pixels and dark pixels before effective data, and the and the horizontal transfer signal output by FPGA to CCD,
active high period T2 corresponds to the pulse widths of over as shown in Fig. 11 and Fig. 12, respectively. Compared
scan pixels and dark pixels. DarkBEn is the enabling of dark with the timing relationship required by the CCD, the
pixels after effective data, and the high active period T3 is horizontal transfer signal and the vertical transfer signal
the pulse width of dark pixels. DataEn is the enable frame for generated by the FPGA can meet the drive requirements of
effective data. the CCD and can drive the CCD to operate normally.
The interface of data transmission circuit adopts LVDS
interface, signal processing circuit outputs image data, clock
signal and line synchronization signal to data transmission
subsystem. The frame synchronization signal is active low.
The rising edge of the clock is aligned with the jumping edge
of the synchronous signal.
D. Telemetry and Telecontrol Module
The telecontrol function refers to the switching of the Figure 11. Phase relationship of vertical transfer signal I1/S1 and I3/S3
working mode by imaging circuit receiving the auxiliary data,
the integral time and the A/D gain through the three wire
serial ports. The telemetry function refers to that after the
imaging circuit receives the three wire clock and enable
signal sent by the management controller, the imaging circuit
sends the status information that needs to be returned to the
management controller. The byte length of the three wire
serial interface data is 32 bits, and the high 8 bits (A7-A0)
are code identification bits, D23-D8 are data bits, and D7-D0
are parity bits. The timing diagram of three wire signal is
shown in Fig. 9.
SDATA A7 A0 D23 D1 D0

SCLK

>1/2T >1/2T
SEN Figure 12. Phase relationship of horizontal transfer signal R1/R2/R3 and
reset signal ØR
Figure 9. Timing diagram of telecontrol three wire signal
The byte width of AD internal configuration register is
IV. TEST AND ANALYSIS 32 bits, which high 8 bits is address, the remaining 24 bits is
data. Because each register corresponds to a different
The software function test scheme of the imaging circuit function, you only need to change the corresponding register.
is shown in Fig. 10. The imaging circuit is connected with The following figure shows the register configuration
the video transfer circuit, and then connected to the upper instruction for sending the address data with X "17".
computer through the video transfer circuit. The video
transfer circuit simulates management controller providing
main clock, synchronization and three wire signal for the
imaging circuit, and receives the data transmitted from the
imaging circuit, converts into the Cameralink data format.
The image data is transmitted to the upper computer through Figure 13. Three wire instruction of AD register
the Cameralink interface, and the image acquisition software
is used to observe whether the image data output is correct The phase relationship between the pixel clock CLI and
on the upper computer. Oscilloscope is used to observe the the analog signal is shown in Fig. 14 below.
timing waveforms of each interface to determine whether the
timing relationship of each input and output signal is correct
[6].
the video
power transfer
circuit
Figure 14. phase relationship between pixel clock CLI and analog signal
Imaging circuit

Using an oscilloscope to observe the data transmitted

Oscilloscope the upper
computer from the signal processing circuit and the waveform of the
synchronization signal as shown in Fig. 15, where the low
Figure 10. Block diagram of test scheme level of the synchronization signal corresponds to valid data,

627
and the data transmission meet the functional and integration design of the imaging circuit is realized. The
performance requirements. integration timing software, focal plane software and signal
processing software in the traditional imaging circuit are
integrated into the same FPGA, which simplifies the design
complexity of the hardware circuit. Through the observation
and analysis of the hardware output signal and the analysis of
the image of the upper computer, the integrated design
Figure 15. phase relationship between clock , synchronization and data method of FPGA software proposed in this paper can meet
the function and performance requirements of the CCD
The series and gain adjustment instructions are sent remote sensing camera on the imaging circuit. It has high
through the upper computer to observe the change of the engineering application value.
image so as to determine whether the instruction response is
correct. The upper computer image is as shown in Fig. 16. REFERENCES
From the top to the bottom, it is an image with a gain of 0dB [1] Da Xuanfu, Wang Huaiyi, and Li Tao, “Design of Electronics System
and a CCD series of 2 level; an image with a gain of 0dB and of CCD Camera Based on Integrated Signal Processor,” Spacecraft
Recovery & Remote Sensing, Vol.29, 2008: 30-35.
a CCD series of 4 level; an image with a gain of 6dB, and a
[2] Song Xiaolong, Zhang Li, and Shi Guangming, “A High Accuracy
CCD series of 2 level. From the comparison of the three TDI-CCD Signal Generator Based on FPGA,” Electronic Science and
images, the gain and the series instruction can respond Technology, Vol.4, 2007: 1~4
correctly. With the increase of the gain and series, the value [3] Ji Lingling, Guo Hui, Zhang Zhifang, and Kang Zheng, “Real-time
of the image data increases and the image becomes brighter. Transmission of Multi-source Information System Based on FPGA,”
Video Engineering, Vol.38, No.11, 2014: 78~81
[4] Xu Jian㸪Hou Zhenlong, Gong Donglei, Fang Ming, “Design and
Implementation of High-speed Serial Data Processing Module,”
Computer Engineering, Vol.42, No.3, 2016: 289~294
[5] Wang Huawei, Liu Bo, and Cao Jianzhong, “Design on CCD Signal
Processing Circuit,” Science Technology And Engineering, Vol.16,
2007: 4153-4156.
Figure 16. The image acquired by the host computer [6] Yu Qingguang, Zhang Xiaoming, and Wang Hao, “Acquisition and
identification system of laser dot from CCD video signal,” Chinese
Journal Of Scientific Instrument, Vol.27, No.6, 2006: 1365-1366.
V. CONCLUSION
Through the integrated processing the software of all
parts of the CCD imaging circuit, the miniaturization and

628