Plan:         Pulsar digital filterbank

Date:                9^th December 2004

Project Leader:   Dick Manchester

1. Project Team

Project Manager:	Grant Hampson
Project Scientist:	George Hobbs
Project Engineer:	Grant Hampson
Customer:	ATUC

2. Solution

2.1. Technical details

The Pulsar Digital Filter Bank hardware will consist of three main parts; a high speed Analog-Digital-Converter (ADC),  a Digital Filter Bank (DFB), and a Pulsar Processing Unit (PPU). This project is concerned with the development of the PPU and associated pulsar timing equipment.

The ADCs will sample at a rate at least twice the desired BW of 1GHz. The ADC will be a multi-bit digitiser for each channel, possibly up to 8-bits of resolution.  The DFB system will have a total of four input data channels; two for data and two for RFI. The DFB will calculate 2048 spectral channels across a selectable BW for each data channel.  Desirable for the DFB outputs to be connected to existing CPSR2 hardware.

The PPU will calculate both auto and cross products for all four data channels to give full polarisation information. The PPU individual pulsar bin timing resolution will be less than four micro-seconds. Synchronous averaging will occur at the pulsar period for all products with full spectral resolution. There will be sufficient memory to store up to 2048 integrated pulsar bins per pulsar period.  Desirable to have multiple parallel PPUs to observe multiple pulsars simultaneously. Double buffered memory for both Search mode and Folded data.

Given that an RFI channel is digitised and processed there is a possibility for RFI mitigation systems to be developed.

2.2. Deliverable

The project deliverable will be a rack containing several cards; a wide band digital receiver 2GSPS@8-bits, a 2048 channel digital filter bank, correlator and pulsar binning integrator, a pulsar event generator and a block control computer.  In addition to the hardware there will be several firmware configuration files available to the pulsar user for re-configuring the hardware.  Software to control the hardware and configurations will also be supplied.

There are two prototypes which will be delivered to Parkes; the first is a Nallatech FPGA board which will be completed in April 2005, and secondly a CABB 16-FPGA board prototype which will be installed in November 2005.  If required, the prototype Nallatech board will be returned to the Electronics Group for further development.

Tables 1&2 list the key hardware/firmware specifications for the Nallatech prototype and final CABB hardware.

Table 1. Nallatech Prototype Hardware - Pulsar DFB Specifications

Specification	Quantity	Comments
IF Inputs	2	IF inputs (called X and Y – typically two polarisations of receive antenna)
Sample Rate	512 MHz	512MSPS of real sampled data
Sample Resolution	8-bits	MAX104 ADC provides ~64dB SFDR
Output Bandwidth	256MHz	512MSPS of real sampled data provides 256MHz of output bandwidth
DFB Frequency Channels	1024	Output frequency resolution is 250kHz (output data rate is 250kSps complex or 4ms per DFB output sample period)
Correlations	4	Direct and cross products: XX*, YY*, real(XY*), imag(XY*)
Data Capture Mode	1Gbyte	For ADC output capture: 2-IFs X 8-bits X 512k–samples = 1Gbyte For DFB output capture: 2-IFs X 1k-channels X 16-bits X 2-complex X 128k–samples = 1Gbyte
Pulsar Timing Resolution	+/-7.5ns	Existing pulsar timing unit (PTU) specifications apply here.
Simultaneous Pulsars	1	Simultaneous PPU Parallel Hardware for monitoring multiple pulsar periods (applies to Folded Data Mode)
Pulsar Bin Integrator (PBI)	1-128	Integrate the 4 correlation products for set fixed amount (Search Mode) or for a variable amount (Folded Data Mode) depending on PTU timing.
Pulsar Bins	2048	Number of pulsar bins per pulsar period in Folded Data Mode
Pulsar Folded Data Mode Memory	64Mbyte	Required memory resources using double buffering: 2-buffers X 2k-bins X 4-products X 1k-channels X 32-bits per integration = 64Mbyte
Pulsar Search Mode Memory	1Gbyte	Required memory resources using double buffering: 2-buffers X 4-products X 1k-channels X 8-bits per integration X 128k-deep=1Gbyte (Modify above parameters (products, channels, bits, and PBI) to get output data rate less than PCI data transfer rate for continuous unbroken data.)
PCI Transfer Rate	~32M bytes per second	64-bit/33MHz PCI Interface (currently using a 32-bit PCI slot, potentially could double the data rate)
CPSR3 Data Outputs	3Gbps	Up to 24-bits LVDS of DFB data at 128MSPS (3Gbits per second) via Nallatech 68-pin shared connector

Table 2. CABB Final Hardware - Pulsar DFB Specifications

Specification	Quantity	Comments
IF Inputs	4	IF inputs (called X and Y – typically two polarisations of receive antenna, and two RFI inputs W and Z)
Sample Rate	2GHz	2GSPS of real sampled data
Sample Resolution	8-bits	ATMEL TS83102G0B ADC provides ~57dB SFDR (This is a 10-bit ADC but likely to be rounded down to 8-bits)
Output Bandwidth	1GHz	2GSPS of real sampled data provides 1GHz of output bandwidth
RFI Processing	Possible	RFI processing possible – subject to additional investigations. May be treated as an additional project.
DFB Frequency Channels	2048	Output frequency resolution is 500kHz (output data rate is 500kSps complex or 2ms per DFB output sample period)
Correlations	4	Direct and cross products: XX*, YY*, real(XY*), imag(XY*)
Data Capture Mode	Memory split into 4-banks of 256Mbyte	For ADC output capture: 4-IFs X 8-bits X 256k–samples = 1Gbyte For DFB output capture: 2-IFs X 2k-channels X 16-bits X 2-complex X 64k–samples = 1Gbyte
Pulsar Timing Resolution	+/-7.5ns	Existing pulsar timing unit (PTU) specifications apply here.
Simultaneous Pulsars	1-4	Simultaneous PPU Parallel Hardware for monitoring multiple pulsar periods (depending on hardware resources)
Pulsar Bin Integrator (PBI)	1-128	Integrate the 4 correlation products for set fixed amount (Search Mode) or for a variable amount (Folded Data Mode) depending on PTU timing.
Pulsar Bins	2048	Number of pulsar bins per pulsar period in Folded Data Mode
Pulsar Folded Data Mode Memory	128Mbyte	Required memory resources using double buffering: 2-buffers X 2k-bins X 4-products X 2k-channels X 32-bits per integration = 128Mbyte
Pulsar Search Mode Memory	1Gbyte	Required memory resources using double buffering: 2-buffers X 4-products X 2k-channels X 8-bits per integration X 64k-deep=1Gbyte (Modify above parameters (products, channels, bits, and PBI) to get output data rate less than PCI data transfer rate for continuous unbroken data.)
Network Transfer Data Rate	~5M bytes per second	Data is transferred from “output” FPGAs via a 32-bit parallel interface to the onboard BCC. The BCC contains an onboard 100Mbit network hardware to connect to correlator PC. To transfer data in the Folded Data Mode (128Mbyte/2=64Mbytes) takes approximately 13 seconds, setting the minimum integration period to 13s. Other options may exist for transferring data in the Search mode such as using the CPSR3 data recorder, VLBI data recorders, serial hard drives, or an embedded 1Gbit network driver in a Xilinx FPGA.  This extra hardware is not taken into account in the time line and additional time and resources will be required. Given that search mode has lower priority it will be considered after all other modes are operating.
CPSR3 Data Outputs	6Gbs	Separate additional hardware (ATCA I/O card) with Rocket IO Tranceivers with conversion interface to parallel LVDS outputs.

2.3. Measures of success

The measure of success for the new Pulsar Digital Filterbank will be the timing precision of known pulsars which are currently observable from the Parkes radio telescope. Given that the hardware is significantly more sensistive with the new digital receiver and filter bank then it should also be possible to use the search mode to locate new pulsars.

2.4. Relation to other projects

Some hardware is common to two other existing and concurrent projects - CABB and MOPS.  Both these projects could take precedence in the case of conflict.

The CABB hardware currently in development is a 16 FPGA cluster with 1Gbyte attached memory.  A prototype board which is currently scheduled to arrive in April 2005, and some time later be available in a chassis for development work. 

The development system for the Pulsar DFB is proposed to be a  Nallatech FPGA platform, which is currently in use as the current MOPRA spectrometer. This system will become available when the milli-metre observing season ends in November 2004. 

Also in development for the CABB system are high speed (2GSPS) multi-bit ADCs which are scheduled to arrive also in April 2005.  The final Pulsar DFB will use these custom ADC’s and the development system will use a commercially available ADC with an existing ATNF interface card.

Research can however commence without both of these items but they will both be necessary to perform hardware integration and testing of algorithms and firmware.

The DFB firmware development for CABB and this project are very similar.  However, the PPU firmware is new and previous pulsar systems indicate several critical areas (asynchronous timing) which may have unpredictable development times.

3. Resources

3.1. Resources

Name	0405 Q1	0405 Q2	0405 Q3	0405 Q4	0506 Q1	Total
Dick Manchester	0.1	0.1	0.1	0.1	0.1	0.5
George Hobbs	0.1	0.1	0.5		0.5	1.2
Grant Hampson	0.5	1	3	1	2	7.5
Technician		1	2	1.5	1.5	6
Dick Ferris	0.1	0.2	0.5	0.2	0.5	1.5
Evan Davis		0.2	0.2		0.2	0.6

Note: 50% of the engineer salary is contributed through Dick Manchester’s Federation Fellowship for the duration of the project.

Item	0405 Q1	0405 Q2	0405 Q3	0405 Q4	0506 Q1	Total
Matlab and toolboxes ($28k)	$14k					$14k
Xilinx FPGA toolset ($10k)	$5k					$5k
Nallatech FPGA modules ($20k)		$10k				$10k
1GSPS ADC (2-channels) ($2k)			$1k			$1k
4GSPS ADC (2-channels) ($10k)			$10k			$10k
Pulsar Timing Unit ($1k)				$1k		$1k
DFB Card ($16k)				$16		$16k
ATCA Chassis ($12k)				$12k		$12k
Clock Sources ($6k)		$1.5k		$1.5k		$3k
Miscellaneous ($6k)					$3k	$3k
Total ($111k)	$19k	$11.5k	$11	$30.5k	$3	$75k

Note: Some of these hardware items are cost shared with other projects. The above table shows the cost to this project, not the total cost of the item.

3.2. Budget

The finance department is currently working on a system to convert the above resource tables into a revenue and expense budget, such as:

Financial quarter	Revenue	Expense
04/05 Q1	$	$
04/05 Q2	$	$
04/05 Q3	$	$
04/05 Q4	$	$
05/06 Q1	$	$
Total	$	$

4. Milestones

1.	Commencement of project.	Oct-03
2.	Arrival of Key Personnel (Grant Hampson)	Aug-04
3	Learn to use FPGA Design Tools on FFT part of DFB	Dec-04
4	Bring HW and SW parts of Mopra Spectrom together and experiment	Jan-05
5	Basic BIN integration on 256MHz DFB	Feb-05
6	Testing BenData DDR SDRAM interface and data rates.	Mar-05
7	Two IFs and folded data integrator implemented	Apr-05
8	Install, Test and Document Prototype Nallatech DFB at Parkes	May-05
9	Initial testing of 16 FPGA board and 2GSPS ADC boards	Jun-05
10	Capturing DFB outputs in SDRAM and transferring through BCC	Jul-05
11	Modify PPU for new multi-output DFB architecture	Aug-05
12	Testing new PPU for all IFs	Sep-05
13	Install, Test and Document Pulsar DFB at Parkes. Project Completion	Oct-05
14	Closure document submitted to PRB.	Nov-05
15	Acceptance of closure document by PRB.	Dec-05

The following is an approximate month-by-month description of the Pulsar DFB. 

Month	Task
August-September	Order PC, Order/Install software (Windows XP, Matlab, Xilinx, etc.) Learning to use tools. Introduction to Digital Filter Banks.
October-November	Experimentation with real and complex FFT implementations. Configuration of Simulink designs. Initial synthesis of designs.
December	MOPRA spectrometer becomes available. Use existing 256MHz spectrometer design (DFB1) Bring all hardware together (PCs, EG, PTU, Nallatech, 8-bit sampler, clock source, test sources, etc.) and software (Boss, 256MHz spectrometer.) Setup and take measurements.  Document the configuration and test results of the MOPRA system.
January	Implement Pulsar DCP (correlator) in Simulink and test in hardware by capturing the auto correlations (only one IF available.) Implement the BIN module by reusing the MOPRA spectral integrator with a fixed number of integrations and a dual memory buffer. Syncronise capture with 1pps and Start/Toggle signals to gain experience with the EG and PTU.  This is Search mode with one double buffered bin. Document BIN and basic Search mode system.
February	Nallatech BenData FPGA/memory module arrives.  Requires the DDR SDRAM controller – test this with dummy data and verify full 1Gbyte read across PCI bus to hard disk and measure overall transfer speed. Partition design so that output of the DFB1 is to the BenData module. Shift BIN integrator to BenData module and capture result for data for verification. Verification by connecting test sources and possibly low frequency antenna. Document BenData system.
March	Get two data channels working by borrowing second BenBlue-II module and connecting second ADC. Extend simple search mode (single bin) to full search mode in BenData memory space and use entire 1Gbyte of memory. Next implement the folded data integrator (FDI) in Simulink and test.  Implement in BenData module and test using a locked pulse generator. Control PTU parameters simultaneously with PPU parameters. Verify the number of integrations per bin and normalise outputs.  Document dual IF FDI system.
April	Time permitting implement channel selection and quantiser for CPSR3 data and send to via parallel cable. Take all hardware to Parkes and setup equipment.  Test system with known pulsar sources and compare with measurements. Document Prototype system with results.
May	The 16 FPGA board and 2GSPS ADC boards become available in ATCA backplane/rack. Plan how each DFB fits into this architecture and split across FPGAs. Plan how to partition the pulsar modules to fit into this architecture and how they all interface to memory and BCC outputs. Document the resulting DFB/PPU partition.
June	2GSPS DFB is available for a limited number of configurations. Implement DFB in Simulink and partition design across FPGAs.  Start testing in hardware by capturing output of each DFB in its memory bank and send results to BCC and across network. Re-use DDR SDRAM core from prototype system to access memory. Measure net data transfer rate. Document initial DFB system.
July	Figure out how to interface the new multi-output DFB to the Pulsar processing unit.  Will need to modify the correlator, BIN and FDI units for multiple inputs.  Have to design architecture that remains synchronized across all parts of the PPU – since it is now distributed across many FPGAs. Document new pulsar processing distribution.
August	Implement new pulsar processing for part of the IFs and test if EG and PTU signals interface correctly into the design.  Partial tests using locked pulse generator. Once working implement remaining  IFs.
September	Implement code to output CPSR3 data across Xilinx Rocket ports to secondary card with conversion to parallel data.
October	Get all the hardware together and install at Parkes. Test system with known pulsar sources and compare with measurements. Document Prototype system with results.