R2DBE: A Wideband Digital Backend for the Event Horizon Telescope

A heterogenous instrument by nature, the EHT forms a VLBI array by adapting existing standalone submillimeter radio telescopes. These include single dish and array facilities. The R2DBE is part of the foundational equipment for the single dish sites, which are listed in Table 1.

Figure 1a shows several of the EHT sites with their projected baselines as seen by Sgr A^∗ at some instant. Figure 1b shows the visibility map over a full 24 hr computed for a wavelength (λ) of 1.3 mm. Black tracks indicate baselines involving pairs of sites used in the past EHT runs: the Submillimeter Array (SMA), CARMA, ARO SMT, PV, and the former Plateau de Bure interferometer (PdB), currently named the Northern Extended Millimeter Array (NOEMA) and preparing for future VLBI capability. Color indicates new baselines to the array produced when adding a one of the three major new sites—LMT, SPT, and ALMA.

Zoom In Zoom Out Reset image size

For continuum emission at submillimeter wavelengths, the signal-to-noise ratio (S/N) of the visibility increases by the square root of the bandwidth, B. This bandwidth is limited on the front end by the receiver and Intermediate Frequency (IF) bandwidth (typically 4 GHz) and at the backend by speed and cost of recording media. Once determined, this bandwidth can be digitized in a number of ways. If using slower analog-to-digital converters (ADCs), the band must be broken into many Nyquist-limited channels. This increases the cost of the analog IF system to deliver the necessary bands at the correct signal levels with appropriate out-of-band rejection.

Our experience has shown that the cost of the analog IF "block downconverter," the component cost of which is not linked to Moore's Law, can dominate the cost and complexity of VLBI backends. Thus we have been motivated to investigate high-rate sampling, and develop new DBE technology to keep pace with the fastest digitizers on the market. This pathway allows the EHT to economically push VLBI bandwidth to new limits, with an end goal of 64 Gbps per station (4 GHz double-sideband, dual-polarization).

There are technical challenges associated with time-keeping. VLBI at submillimeter wavelengths requires stability on the order of a few tens of femtoseconds on 10 s timescales (Thompson et al. 2007), thus installation and maintenance of a hydrogen maser at every EHT site are required. Additionally, with these extremely high data rates, accurate time-stamping of the data is necessary to reduce time spent at the correlator searching for initial correlations.

Logistical challenges exist with submillimeter DBE deployment and maintenance. Sites are often in remote locations, and accessibility can impose constraints on the digital design process. In the case of the SPT, shipping of all hardware had to be completed by early 2014 fall to guarantee its installation by 2015 March. This placed a tight schedule on the hardware choices, but by utilizing open source platforms, commercial-off-the-shelf (COTS) products, and field programmable gate arrays (FPGAs), engineers were able to ship hardware that met validation requirements, while firmware upgrades continued and continue to be developed, tested, and pushed to the EHT array.

VLBI digital backend equipment for geodesy and astrometry communities is guided by the standards and goals set out by VLBI2010 (Niell et al. 2005), and a comparison of these systems is given in Petrachenko (2013). Among these include systems sampling over 2000 MSps, though these have not been utilized by the EHT in the past: the Chinese Data Acquisition System (CDAS) (Zhang et al. 2008; Chen et al. 2009; Zhu et al. 2010), and the ADS3000+ system developed by the National Institute of Information and Communications Technology (NICT) (Takefuji et al. 2010).

For astronomical VLBI, specifically in past EHT experiments, two families of DBE have been used: the DBBC (Digital Baseband Converter) and the RDBE (ROACH Digital Backend). The DBBC series is a project from the European VLBI Network (EVN). The most recent model used in the field for EHT experiments is the DBBC2 (Tuccari et al. 2010). The DBBC3 (Tuccari et al. 2014) is a system still in development, but when fielded will sample at 8192 MSps. Both these systems provide the user many bitcodes and modes of operation.

The R2DBE is a next generation of the ROACH digital backend (RDBE) (Neill et al. 2010), which has been used for past EHT experiments and is currently in use at the VLBA.⁵ The RDBE system comes in a family of platforms with slight hardware modifications, but all RDBEs use the iADC⁶ and the ROACH board. The RDBE-S was designed with the sole purpose of doing single channel recording and was parallelized to create a 16 Gbps digital backend system (Whitney et al. 2013), much like we propose to do with the R2DBE up to 64 Gbps. We present details on the RDBE-G running software PFB v1.4 for comparison, as it is a standard VLBI digital backend mode used in the field presently, and is the mode we used for commissioning the R2DBE, described in § 4.

The R2DBE produces real-valued, time domain, or "single channel," data. The 2.048 GHz chunk of bandwidth can be segmented in the correlator software to match any filterbanked data product. At 4096 MSps, the R2DBE extends the sampling rate of the RDBE-G PFB v1.4 digital backend by a factor of 4, and transmits eight times more data to the recorder.

We select the single channel mode of operation as it is expected to serve the EHT currently and in the near future. We compare DBE technologies in Table 2. Up-to-date information on the DBBC systems was provided in A. Roy & G. Tuccari (2015, private communication).

Submillimeter telescopes use a superheterodyne receiver architecture to coherently downconvert the band of interest captured by the antenna from 230 or 345 GHz down to a typical IF range of 4–8 or 5–9 GHz. There are three major stages involved in the digital backend pipeline: an analog system to condition the IF signals for sampling; a digital backend that samples the signal, performs necessary digital signal processing, formats and time-tags the data; and a data recorder that shores the data for subsequent shipment.

An IF downconverter, designed at MIT Haystack Observatory, mixes either 4–8 and 5–9 GHz IF input (expected IFs for 230 GHz and 345 GHz, respectively) in two polarizations, to two 2 GHz baseband blocks suitable for analog-to-digital conversion. The unit is customizable, and can be used for 32 Gbps operation at two different IF ranges, or 64 Gbps for one of these frequency ranges in double-sideband mode.

The R2DBE samples the two 2 GHz baseband chunks at 4096 MSps on separate inputs and ADCs. The data are requantized to 2-bits based on a 1-sigma threshold estimated from the histogram of 8-bit samples. The samples are time-tagged and formatted using the VLBI Data Interchange Format (VDIF) (Whitney et al. 2009). The data packets are then transmitted via two 10 GbE ports to the data recorder.

The EHT uses the Mark 6 data recorder system jointly developed by MIT Haystack Observatory, NASA/GSFC High-End Network Computing group, and the Conduant Corporation. The data stream from the R2DBE flow into two 10 GbE inputs on the Mark 6 and expansion unit for a total of 16 Gbps. Details of the Mark 6 VLBI system and a history of VLBI data recorders can be found in Whitney et al. (2013).

In addition, peripheral equipment is needed to support the operation of the R2DBE, such as a 2048 MHz synthesizer and a noise/IF distribution system. The latter will switch between delivering the outputs of the IF downconverter as in the case of standard VLBI operation, or correlated wideband noise for diagnostics, to the inputs of the R2DBE.

Figure 2 shows an infographic of the complete 16 Gbps digital backend system based on the R2DBE, used in the field for the EHT in 2015 March, and offers a suggested pathway to 64 Gbps using existing equipment. Some of the peripheral equipment supports more bandwidth natively. Each EHT station can scale to a 64 Gbps station in the near future by parallelizing existing equipment.

Zoom In Zoom Out Reset image size

Working with the CASPER ROACH2 board facilitated the adoption of fast samplers. Academia Sinica Institute of Astronomy and Astrophysics (ASIAA) has developed ADC boards (Jiang et al. 2012) built on the e2v EV8AQ160 Quad ADC,⁷ capable of data conversion up to 5 GSps with 8-bit precision. This board provides the FPGA with demultiplexed data in 16 parallel streams, the FPGA clock rate (1/16 the sampling rate) and a pulse-per-second (PPS) signal, often provided from a Global Position System (GPS) receiver, sampled at the FPGA clock rate. The IF inputs to the R2DBE should not have power higher than -1 dBm; however, an optimal input level of -7 dBm at the R2DBE assembly input corresponds to -13 dBm at the ADC input. This is the appropriate level to optimize the Noise Power Ratio of the ADC, as shown in (Patel et al. 2014) Figure 9. The performance of the ADC degrades above Nyquist Zone 1, thus we restrict our design to baseband sampling. The frequency response in the passband has a linear slope in log-power, resulting in 6 dB loss near 2.15 GHz. According to Rogers (2010), we can expect the estimate of visibility amplitude to incur no inherent loss (unlike quantization); however, we lose roughly 5% in S/N of this estimate because of this slope, when both sites operate in the same sideband. The quad-core misalignment of the ADCs did not dramatically affect input power, and the spurious signals produced do not correlate from site to site due to Doppler shift. Thus, core-to-core alignment was not a top priority in 2015. That being said, scripts were developed and distributed to sites to perform automated core-to-core calibration prior to the VLBI window, and we found this one-time site calibration remained stable over the course of the 10-day campaign. Characteristics of this ADC board and improved performance have been discussed in the literature (Patel et al. 2014).

To match standard VLBI sampling rates which are generally radix-2 in MSps, a sampling rate of 4096 MSps was chosen, setting the Nyquist frequency at 2048 MHz. The clock signal input to the R2DBE (0 dBm sine wave) drives both the sample clock and the FPGA clock. The clock signal is required to be 2048 MHz, exactly half the sampling rate desired, resulting in the FPGA clock speed of 256 MHz.

The R2DBE has an output port for signal monitoring. In version 1.0 of the R2DBE software, this port provides the R2DBE PPS (square pulse, 10% duty cycle), which can be compared on a scope to any external PPS (such as GPS or maser) for diagnostics. In future versions of the bitcode, other 1-bit signals could be routed here through a demultiplexer with software-controlled selection, so during operation different diagnostic signals can be investigated.

After the data are processed (details in § 2.3), they are formatted into 8 kB VDIF packets and transmitted out two 10 GbE ports on one of the two 10 GbE SFP+ mezzanine cards.

Since the ROACH-2 and its enclosure are open source, the R2DBE utilizes a subset of available ports on the front and back panels. Figure 3 shows the enclosure of the 1U R2DBE and briefly notes which ports are in use and for what purpose.

Zoom In Zoom Out Reset image size

The digital backend executes real-time data processing via a bitcode (also known as firmware or gateware) running on a Xilinx Virtex-6 FPGA (XC6VSX475T).⁸ The bitcode described here was used in 2015 March and is referred to as the 1.0 bitcode. New bitcodes can be developed and remotely distributed to the EHT sites.

Bitcode development was done using MATLAB^® and Simulink^® version 8.0, R2012b (The MathWorks Inc., Natick, MA, 2012), in addition to hardware description language (HDL) code provided by Xilinx and CASPER. Using the CASPER tool flow framework the system is designed in the Simulink environment by building logic blocks that invoke underlying HDL. Thus, the code is developed as a block diagram. Figure 4 shows a functional overview of the R2DBE in the style of the top-level Simulink diagram.

Zoom In Zoom Out Reset image size

The sampled IF data and external PPS signal from each of the two ADC boards are presented via TE Connectivity (Formerly Tyco Electronics) Z-DOK+⁹ connectors to the FPGA as 8-bit signed integers in 16 parallel streams. The logic for this data transfer is contained in modules labeled zdok_0 and zdok_1.

The onepps block takes the external PPS signal in from both ADC boards (labeled the sync signal). If a maser PPS is present and connected to the correct input port, the R2DBE will monitor this signal for period (should be an unwavering 256 million FPGA clock samples) and total number of pulses. The GPS PPS should be connected to the R2DBE as described in § 2.2. In addition to computing statistics like period and total pulses, the R2DBE uses the rising edge of the GPS PPS to "arm" a digital PPS. The R2DBE maintains this internal digital PPS whose period is derived from the FPGA clock, which is locked to the maser, and whose rising edge is synchronized to real-time by this one-time "arming" operation. This internal PPS is routed to the rest of the design as well as to the smon port for signal monitoring.

Another diagnostic tool built into the bitcode is the data multiplexer, labeled data_mux. During R2DBE operation, a software register controls which signal passes through this block to the processing code. By default, it routes the 8-bit IF data. However, digital noise and tone are available options.

There is a random delay between the two 8-bit data streams resulting from division of the sampling clock inside the ADC. The phase of the internal divider is established randomly at power-up, and while this can be removed at the correlator, it is useful to be able correct this offset on-site by delaying one stream relative to the other an integer number of samples. Immediately preceding this delay the FPGA fills a memory buffer with the 8-bit data, shown as the snap_8bit block. The software provided for R2DBE v1.0 provides a method for automatic correction of this unknown delay by a simple process executed in Python:

• 1.  
  The Noise/IF Distribution box is remotely controlled to send correlated noise into the two inputs of the R2DBE.
• 2.  
  The 8-bit data snapshot for each stream is captured and correlated.
• 3.  
  The peak is located in lag-sample space.
• 4.  
  This integer sample delay correction is written to a software register in the delay block of the relevant stream.
• 5.  
  The Noise/IF Distribution unit is programmed back to IF Distribution mode.

The data are then quantized to 2 bits, according to an 8-bit threshold that the user can set and change during operation. We provide automatic level control algorithms that can be run on the control computer to set this value near 1-sigma deviation at the start of every scan. Following this there is another collection of data, thus the 8-bit and 2-bit quantized data can be directly compared.

The data are then packed into VDIF frames, which requires some data manipulation to meet the standard (endian byte-swap, sample order matches VDIF spec, etc.). The VDIF header includes metadata such as the maser-gps offset, the type of data, length of the packet, and most importantly, a precise time-stamp. At runtime, the control computer will provide a reference epoch identifier and a "seconds since the epoch" integer to the R2DBE which is then stored in memory, stamped in the VDIF header, and incremented every rising edge of the digital PPS. Counters on the FPGA increment with each VDIF packet that has been streamed, resetting the count every PPS edge. For the R2DBE, the data are packed into frames containing 8μ seconds of data. Once the data are recorded to disk as a VDIF file, the correlator operators can run a script to recover the metadata in the file, including a precise time-stamp found from the reference epoch, the seconds since that epoch, and the frame number of the packet within that second.

These packetized data streams are buffered and transmitted out separate 10 GbE ports to the Mark 6. With this single channel processing scheme one R2DBE can send a 2 GHz, dual-polarization experiment to a data recorder at a rate of 16 Gbps, or 2 GBps.

Also of important note is that the R2DBE v1.0 bitcode uses few FPGA resources as summarized in Table 3. FPGA design is challenging as large utilization leads to difficulty achieving timing closure for the bitcode. The R2DBE v1.0 bitcode compiles in under 2 hr, with no need for floorplanning, the manual placement of primitives on the FPGA to logically assist in achieving timing specifications. The simplicity of the design allowed for fast development and focus on testing. The excess resources allow us to explore the inclusion of features such as adaptive equalization and polyphase filter bank in future generations.

The R2DBE is controlled and monitored from a separate computer sending commands via gigabit ethernet to an on-board PowerPC 440EPx. Our standard configuration uses Mark6 data recorders as the control computer, providing a network file system (NFS) for the R2DBE containing all available bitcodes. Python scripts executed on the computer initialize the R2DBE, control the Noise/IF Distribution box, run diagnostics, and set and query software registers and memory that store state information on the FPGA. Particularly useful information is the maser-GPS offset, an estimate of the FPGA clock speed (an on-chip clock counter queried and compared to the Mark 6 system clock), and short snapshots of 8-bit and 2-bit quantized data.

At EHT wavelengths, telescopes are located at altitude to mitigate atmospheric effects. Altitude can be a challenging environment for operators, thus equipment must be straightforward to use. Several manual operations were converted to automatic scripts run on start up, such as automatic optimization of the 2-bit quantization threshold, and extensive documentation and user guides were developed. Diagnostic tools were developed such that users would be alerted to suboptimal or incorrect operation. Figure 5 shows the graphical output of a monitor script. The annotations aid the operator in diagnosing power levels, quantization thresholds, that the cables are connected properly, and verify and monitor the drift from real time. Software and firmware updates are distributed electronically. Each site in the EHT can pull from the git repository,¹⁰ for the latest bitcodes and software, giving us a pipeline for continued improvement of our equipment after hardware installation.

Zoom In Zoom Out Reset image size

The experiment was designed such that we provide measured amounts of correlated signal to the two inputs of the R2DBE. Producing semicorrelated noise requires three independent noise sources and a series of splitters and combiners. Measuring the amount of correlated signal as a fraction of total power entering each input of the R2DBE required couplers which directed a small amount of the signal power (20 dB down) to a power meter. Thus, by varying attenuators after the noise sources, we can vary the correlation coefficient of the two analog data streams at the input of the R2DBE and measure these values carefully without disconnecting the apparatus.

While this limits the extent to which component impedance mismatch changes between different correlation coefficient values, reflections nonetheless occur, and can limit our dynamic range. Judicious use of inline attenuators or "pads" attenuate reflections and crosstalk. We also chose components with good isolation across 0–2 GHz and use filters to limit the signal frequency extent to within limits of these components.

Unlike the EHT experimental data where each site has a different Doppler shift, this zero baseline correlation is vulnerable to the spurious frequency spikes produced by the ADC quad-core mismatches causing inflated correlation coefficients in the digital data. Calibration of the core offsets is done in the lab and the alignment, while still not perfect, greatly reduces this effect. Thus we are able to extend the experiment to a lower limit of 0.025, with an upper limit of nearly 0.2 set by the maximum power available at our noise sources.

Figure 6 shows this setup including component part numbers. The attenuators on N1 and N3 are turned completely up allowing us to measure the power of the correlated noise source, N2, at points A and B, and thus extrapolate the amount of correlated power at if0 and if1. The attenuators on N1 and N3 are then adjusted to bring the power at if0 and if1 up to a constant, near the optimal -7 dBm level. Thus, we obtain an estimate of a ratio of correlated and uncorrelated noise at each input and can calculate an analog correlation coefficient. We repeat this process with varying levels of correlated power (maintaining near constant total power) to investigate the analog correlation coefficient space.

Zoom In Zoom Out Reset image size

These two analog data streams entering ports if0 and if1 of the R2DBE will then be sampled by the separate ADCs. Snapshots available on the R2DBE fill with a copy of a quarter of a million time domain samples every second (using the leading edge of the internally generated PPS). The data then pass through the 2-bit quantization block, and a copy of the 2-bit samples also fill a snapshot block. These snapshots can be read off over the 1 GbE ethernet and processed.

The 8-bit data are cross-correlated and an estimate of correlation coefficient is made. The 2-bit data are similarly processed. We explored nine different values of analog correlation coefficient, each time taking 10 s of data (roughly 2.5 million samples) each at 8-bit and 2-bit precision. Before each of the nine data acquisitions, we ran the automatic level control algorithm designed for use in the field with the R2DBE to set the 2-bit quantization threshold to the level closest to 1 - σ of the Gaussian distributed 8-bit samples. Thus, we try and replicate field operation to learn if there are any shortcomings of our system.

The 2-bit quantization of the data is predicted to incur a loss of roughly 88% of the correlation amplitude (Cooper 1970) (only valid in the range of correlation coefficient values tested here) when mapping the ± σ-threshold quantized data to values -3, -1, 1, and 3. Thus, we apply this mapping scheme to the 2-bit digital data prior to computation of the correlation coefficient for direct comparison, and while we use a full floating point estimate of correlation coefficient as opposed to the 2-bit correlator described in Cooper (1970), the difference between these methods is so small that it does not affect the comparison of the results.

In Figure 7, we plot the correlation coefficient into the R2DBE as measured in the analog domain on the x-axis, and the correlation coefficient as calculated digitally on the y-axis. We add the ideal slope 1 line showing that if there is no effect on the data that the correlation coefficient of the input data streams should equal the correlation coefficient of the digital data streams. For analysis, we also include the curve predicted by Cooper (1970) for the 2-bit quantization loss. Finally, we plot the measured data points, an 8-bit and 2-bit estimate for every measured analog correlation coefficient level we investigated.

Zoom In Zoom Out Reset image size

The error bars are quite small, thus we zoom in on a select pair. Horizontal error is simply the noted range of values produced by the power meter. Vertical error is an estimate of the error in the computed correlation coefficient from digital data. The uncertainty is strongly reduced by the fact that we processed many seconds of digital data coherently. The mean error in the y-direction is +1.23% for 8-bit data, and +1.24% for 2-bit data. If there is an offset between the digital correlation coefficient derived from the 8-bit data, nearly the same offset is present in the 2-bit data estimate. This agreement is apparent in the data set and can be seen clearly in the zoomed pair of data points.

In fact, if we want to isolate the performance of just the 2-bit quantizer stage, we can compare the 2-bit correlation estimate to an expected 88% of the 8-bit estimate (rather than 88% of the expected ideal case). This analysis shows excellent performance of the 2-bit quantizer, with an average error of 0.01% from this adjusted expected value. Additionally, one can see that the estimated correlation coefficient inflates toward the bottom left of the plot. Here, we are reaching the lower limits of our experiment and are unable to decouple the additional correlation at zero-lag due to the remaining core misalignment in the ADCs, specifically a tone at 1024 MHz, and symmetric reflections in the system.

The agreement between expected and actual correlation for 2- and 8-bit case gives us confidence that we understand the effects of sampling, the transfer function of the ADC board, and the quantization scheme, and that these have little effect on the correlation amplitude of the signals produced. However, there are notable differences between this experiment and VLBI that affect the correlation amplitude. First, the lab setup has natural limits such as the alignment of ADC cores and reflections due to component mismatches. These affect zero baseline correlations but are not a problem for VLBI where we correlate data from disconnected, Doppler-shifted sites. Second, the lab experiment was limited to a regime of correlation coefficient that we are unlikely to see in the field.

§ 4 describes a VLBI experiment which shows an estimated correlation amplitude on an astronomical source of roughly 3.4 × 10^-3, an order of magnitude smaller than our laboratory limitations. Actual sky tests using radio telescopes do not give us the control we are afforded in a carefully arranged laboratory apparatus. Thus, the field test presented in § 4.1 and 4.2 is complementary to the laboratory testing and necessary to validate the R2DBE for EHT operations.

The RDBE-G was configured identically at both Westford and GGAO to digitize dual linear polarization from 5988 to 6500 MHz. The updown converter was set to deliver the band of interest, preserving frequency order, to another IF of 512–1024 MHz. As the RDBE-G samples at 1024 MSps, this direct sampling of Nyquist Zone 2 produces a reversed spectrum in the digital domain.

The BDC used in the R2DBE signal path converts 5000–7000 MHz lower sideband down to baseband, reversing the frequency order of the spectrum. The R2DBE then samples this Nyquist Zone 1-limited signal. At 2-bits per sample, the output rate is 8 Gbps into a Mark6 data recorder. The full 32 Gbps system was exercised, but only the two polarizations in this frequency block overlapped with the RDBE-G setup.

The quasar 4c39.25 was chosen as it was bright and conveniently observable during the daytime. The setup at the Westford Radio Telescope is presented in Figure 9. The lower pathway for the RDBE-G is identical to the experimental setup at GGAO.

Zoom In Zoom Out Reset image size

Cross-correlation produced successful detections of 4c39.25 for all pairwise combinations of digital backends in the experiment.

The 2-bit data stored on the Mark6 system is first correlated using DiFX (Deller et al. 2011), a frequency domain software correlation package. Both cross-polarization and copolarization correlations are computed for all three pairs of units. With the missing polarization for the R2DBE, this reduces to eight correlations. These eight correlator outputs are then processed by fourfit, a fringe-fitting software package developed at MIT Haystack and distributed with the Haystack Observatory Postprocessing System (HOPS).¹¹

Correlation of a single channel data product with a channelized data product, such as this R2DBE versus RDBE-G correlation, requiring use of the DiFX "zoom" band option, was atypical for past EHT processing. Knowledge of this feature and its usage had to be developed to partition the R2DBE channel to match the eight 32-MHz channels of the R2DBE.

An illustrative result is the 10 s integration of the strong, linear copolarization detection on the Wf-GGAO baseline for both the geodetic and cross-unit tests. These detections are often called "fringes" in VLBI, and a positive detection manifests in delay or delay rate space as a sinc function.

The fourfit output is presented in Figures 10 and 11. Figure 10 compares the amplitude of the fringe found in the control test and the cross-unit test. The amplitude is produced in units of 10^-4 and is presented here plotted as a function of one of the four parameters: delay rate. The fringe amplitude was found to be 3.37 × 10^-3 for the control (shown in Fig. 10a) and 3.49 × 10^-3 for the cross-unit test (shown in Fig. 10b). These numbers show excellent agreement.

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Figure 11 shows amplitude (blue, connected points) and phase (red, disconnected points) of each of the eight 32 MHz sub-bands (labeled a–h) involved in correlation, computed every 0.4 s and displayed as a function of time for the 10-s integration period. The rightmost block shows the vector sum of all eight sub-bands. Again we plot the control experiment in Figure 11a and the cross-unit test in Figure 11b. These plots demonstrate that a strong and consistent fringe was detected in each sub-band, and in each segment of time. Apart from residual phase offset, comparison of these plots shows excellent agreement between the R2DBE and the RDBE-G data products over the band analyzed.

In setting up the system, care must be taken to ensure that the R2DBE is properly synchronized to a low jitter PPS from a reliable GPS receiver. Synchronizing to a particular pulse that is out of phase from the rest of the EHT array (arising, e.g., by jitter on the PPS signal, poor satellite visibility at the GPS receiver, spurious signals on the PPS signal wire, etc.) can cause difficulties finding initial correlation. Due to the large data rates the EHT desires and that the R2DBE provides, logistically the fringe search window must be restricted to a few microseconds. Thus it is vital that the backend time stamp is accurate, and any drift of the clock (derived from the maser) compared to real time is accounted, and can be added a priori.

The agreement of amplitude measurements, and the consistency of amplitude and phase over time, assured us that the R2DBE IF and digital data pathway is functioning properly and produces fringes when correlated against a known, working unit. This test was only over 512 MHz of the 2048 MHz bandwidth of the R2DBE, but shows that the major elements necessary for reliable VLBI backend operation are correct. The complementary laboratory tests described in § 3 validated the R2DBE full bandwidth and quantization, showing that a signal passing through this system incurs only the expected and quantified correlation and losses.

The R2DBE met with other initial success in the field. Subsequent tests in 2015 January validated 1024 MHz of the band with another existing and field-tested unit, the DBBC2, in early 2015 January. In that same experiment, a fringe from the R2DBE to phased ALMA was found exercising the full 2048 MHz of the R2DBE band.¹² The R2DBE achieved further success with initial fringes detected between the SPT and APEX, each with an R2DBE system as the digital backend.¹³

There are a number of enhancements possible through updates to the first release bit code for the R2DBE. A bulleted list is given, followed by brief descriptions.

• 1.  
  Traditional multichannel PFB for VLBI in environments with interference
• 2.  
  Passband equalization for improved sensitivity
• 3.  
  Improved 8-bit to 2-bit quantization
• 4.  
  Built in maser drift measurement
• 5.  
  Operations improvements such as logging, automation

Implementing a traditional multichannel PFB would allow our system to function similarly to other VLBI backends, and open up the R2DBE for use in other fields, such as geodesy. This would also address passband equalization, as it would be implemented before the 2-bit quantization.

Passband equalization can also be achieved using a compensating filter, in either the analog or digital domain. Between the IF downconverter and the R2DBE input, a filter with positive gain slope across the 0–2 GHz band can be added to pre-emphasize the signal such that the band is flat after the ADC. To truly achieve a flat band, the individual EHT sites must maintain flatness of the band into the IF downconverter, which may require attention and specifications unique to stations.

Alternatively, a whitening filter can be implemented digitally, which has the added bonus of being adaptable. The parallel nature of the data (demultiplexed by a factor of 16) as it propagates through the FPGA suggests this be implemented by the overlap-add or overlap-save methods, with finite impulse response (FIR) filter coefficients calculated and loaded with software. Either of these analog or digital methods of passband correction maintains the single channel nature of the R2DBE v1.0 data product.

Better 2-bit quantization can be performed to remove asymmetries in the mapping of 8-bit to 2-bit samples.

One calculation needed for correlation of VLBI data is an estimate of maser drift. This is often done by a separate hardware system. However, the R2DBE tracks the offset between the GPS PPS and internal PPS (maser), and could make routine estimation of drift over time a deliverable parameter.

Along with drift measurements, other information is useful to collect at the site. Operator burden can be reduced by more extensive logging done by the R2DBE and automation of tests, such as a zero baseline test involving Mark6 data recording.

Despite these potential improvements, the R2DBE met with much initial success. The system was tested in a laboratory environment, and on the sky, validating its performance against other units fielded for VLBI.

We have shown clearly the pathway to 64 Gbps digital backend processing for the EHT is within reach.

The 32 Gbps system shipped to the South Pole in early November, our first of many R2DBE systems deployed for 2015 March. The development and commissioning of the R2DBE unit happened in just a few short months, illustrating how toolflows like the CASPER collaboration's package can greatly reduce time-to-science.

We gratefully acknowledge support for this work from the Gordon and Betty Moore Foundation (GBMF-3561). The development has benefited from open source technology shared by the Collaboration for Astronomy Signal Processing and Electronics Research (CASPER), supported by NSF grants AST 0906040 and AST 1407804, Collaborative Digital Instrumentation for the Radio Astronomy Community. Special thanks to Robert (Bob) Wilson who made valuable contributions to the laboratory work.