The High Time and Frequency Resolution Capabilities of the Murchison Widefield Array

Abstract

The science cases for incorporating high time resolution capabilities into modern radio telescopes are as numerous as they are compelling. Science targets range from exotic sources such as pulsars, to our Sun, to recently detected possible extragalactic bursts of radio emission, the so-called fast radio bursts (FRBs). Originally conceived purely as an imaging telescope, the initial design of the Murchison Widefield Array (MWA) did not include the ability to access high time and frequency resolution voltage data. However, the flexibility of the MWA’s software correlator allowed an off-the-shelf solution for adding this capability. This paper describes the system that records the 100 μs and 10 kHz resolution voltage data from the MWA. Example science applications, where this capability is critical, are presented, as well as accompanying commissioning results from this mode to demonstrate verification.

1 INTRODUCTION

The canonical data path through a radio interferometer includes cross-correlation, since this provides a sampling of the spatial coherence of the sky brightness distribution on scales commensurate with that of the interferometer baseline distribution. These visibilities are required in order to reconstruct the image plane from the observed data (Thompson, Moran, & Swenson 2007). Since this process alone increases the data volume by a factor of (N − 1)/2, where N is the number of antennas, time averaging is invariably included into this operation as a way of increasing the signal to noise of the visibility set. This integration can continue as long as the visibilities remain coherent, and significantly reduces the data volume (typically 4–6 orders of magnitude). Obviously, phenomena on timescales shorter than the correlation integration time (typically on the order of one to a few seconds) will be smeared or even reduced below detection through this averaging. While sources varying on these timescales (e.g. pulsars, fast transients) have typically been observed by single-dish radio telescopes (e.g. Parkes, Green Bank Telescope, Arecibo) the increase in resolution available with present-day interferometers argues strongly for their use to observe these exotic astronomical objects. Looking ahead toward the future of radio astronomy instrumentation, which is focused on even larger, more elaborate instruments such as the constituents of the Square Kilometre Array (SKA, Dewdney et al. 2013) where interferometric sensitivity will surpass that of single-dishes, implementing high time resolution observations in the SKA precursor instruments seems an obvious step.

The MWA (Tingay et al. 2013a; Lonsdale et al. 2009) is a new low-frequency (80–300 MHz) radio interferometer located, roughly 600 km north of Perth, at the CSIRO Murchison Radio-astronomy Observatory (MRO) in Western Australia which has recently entered regular operations as the low-frequency precursor to the SKA. Its four key science themes are: (1) statistical detection of the epoch of reionisation; (2) galactic and extragalactic astrophysical processes; (3) solar, heliospheric and ionospheric imaging, and analysis; and (4) time domain astrophysics (Bowman et al. 2013). The standard MWA signal path typically generates visibility cubes from the correlator at a cadence of 500 ms from 24 × 1.28 MHz wide bands with a frequency resolution of 40 kHz. Details of the correlator implementation are described at length in Ord et al. (2014). While there exists some flexibility in the integration time and spectral averaging within the correlator, the data rate must be limited by what can be sustainably transferred into the archive.

This paper describes a new system, which enables recording of the entire channelised voltage input to the correlator, preserving the full instrumental resolution in both the time and frequency domains. This system is called the Voltage Capture System (VCS). We also present example science use cases, for which the VCS capability is critical, together with the results of commissioning observations that demonstrate performance in these science areas. The VCS mode was offered for the first time as part of the 2015-A MWA observing semester (January–June, 2015) and five proposals that focused on this mode were received, ranging from solar science to FRB and pulsar studies, to instrument verification, confirming the demand for such a system at the MWA.

2 VOLTAGE CAPTURE SYSTEM

2.1 Data capture

The standard MWA signal path, as described in detail in Tingay et al. (2013a), Prabu et al. (2014), and Ord et al. (2014), can be simplified to the following synopsis. Dual-polarisation dipole antennas arranged as 4 × 4 grids (tiles) are analogue beamformed, with the radio frequency signals sent to receiver boxes in the field. Each of the 256 signals (128 tiles × 2 polarisations across 16 receivers) is subsequently amplified, digitised, and processed through a coarse polyphase filter bank (PFB) in the receiver enclosures. The outputs from each receiver are then sent to one of four fine PFBs (1/4 of the tiles to each). The 32 fine PFB outputs are then converted from RocketIO (a Xilinx serial protocol) to transmission control protocol by 16 media converter servers (CISCO UCS C240) using peripheral component interconnect express mounted cards produced by Engineering Design Team Incorporated (EDT Cards). These 32 signals are then reorganised into coarse channel groups and sent to the 24 servers hosting the graphics processing unit (GPU) cards that perform the cross-multiply step of correlation. After correlation, the output is transferred offsite and archived.

The initial digital system design did not include any access to the voltage data along this path. We decided to implement voltage capture on the media conversion servers since they otherwise have a relatively small workload and had not been purchased or even specified at the time of this subsystem’s design. The media converter servers were subsequently specified to include 128 GB of RAM each, for potential buffering purposes, 24 × 2.5′′ HDD slots, and a dual-channel drive controller. The digital signal path, receivers through GPU servers, are schematically represented in Figure 1 to highlight where the voltages are recorded.

Figure 1. Simplified illustration of the MWA digital signal path. For the VCS mode, the baseband data between the fine channel polyphase filter bank (PFB) and the X-engine of the correlator are written to local RAIDs on the 16 media converter servers (highlighted with a dashed box). This gives us 100 μs resolution and frequency channels 10 kHz wide.

Each server is equipped with two redundant arrays of independent disks (RAIDs) for recording voltage data. These are each comprised of six 2.5′′ 300 GB (279 GB actual) 10K SAS drives and are combined as a hardware controlled RAID 0 (block level striping but without parity or mirroring), giving each RAID 1.44 TB of usable storage with no redundancy. The RAIDs are on independent 6 GB s^{− 1} channels in the controller, to maximise data throughput.

The 4-bit + 4-bit complex voltages from the fine PFB stream through the media conversion servers at a rate of ~ 4.2 GB s^{− 1}. After being converted by the EDT cards, these data can be streamed directly to the RAIDs. Each server has two PFB lanes passing through it, each of which is directed to a separate RAID in the unaltered post-PFB format as files on 1 s boundaries. Therefore, every second of VCS observation generates 32 × 242 MB files across 16 machines (where each file contains 1/8 of the fine channels for 1/4 of the tiles) for a combined rate of 7.744 GB s^{− 1}.

It is important to highlight that the ‘original’ digitised tile voltages are not recorded, instead it is the channelised output of the four PFBs (i.e. the data have been aggregated, filtered, amplified, and processed through two PFBs before being recorded).

2.2 Capabilities

VCS Observations consist of 30.72 MHz of bandwidth, in 1.28 MHz wide subbands, spread as desired between 80 and 300 MHz. These data have 10 kHz channels and 100 μs time resolution (corresponding to critically sampled 10 kHz channels at the Nyquist Shannon rate). These data are the same as the standard correlator input. At most, the system can record a single uninterrupted time span of ~ 100 min. Dedicated hardware, in the form of a server with 16 × 3.5′′ SAS-drive bays, is installed in Perth where the recorded data can be transferred onto a software RAID utilising a 10 GB s^{− 1} link to the MRO. Significant time, however, is required between subsequent VCS observations to transport the data from the site since archiving correlated observations takes precedence.

Recording the data in this mode gives the user maximum flexibility in processing the data, in particular how the 256 data streams (polarisations × tiles) are combined. The simplest merger of these data is the incoherent sum, where the voltages from each tile are multiplied by their complex conjugate, to form the power, and subsequently summed. This preserves the field of view (FoV) while increasing the sensitivity over a single tile by a factor of $\sqrt{N}$, where N is the number of tiles summed. All the data presented in this paper were combined in this fashion unless noted otherwise.

Alternately, more than an order of magnitude increase in sensitivity can be gained if a phase rotation is applied to each voltage stream before summing the voltages to form a coherent beam at the cost of a drastic reduction in the FoV for an individual phase centred beam. To attain the same sky coverage as the incoherent sum, thousands of coherent beams need to be processed. Since these data are recorded from the ‘standard’ data path, it is also possible to cross-correlate these data offline in a manner similar to normal but with control over the temporal and spectral integration (within the constraints of the raw data and the available compute resources).

3 VCS COMMISSIONING RESULTS

As the VCS mode was added to the MWA, a wide range of engineering tests and on-sky commissioning data were taken to verify the elements as they were added as well as for data pipeline development. These data, taken with a variety of bandwidths and number of tiles, are presented here to demonstrate the capabilities of the instrument. VCS observations in this paper are labeled YYYYMMDDX where YYYY is the year, MM is the month, DD is the day of each observation, and X is an incremental letter denoting the order of observations for the day (i.e. A, B, C,. . .).

3.1 Pulsar science

Arguably the primary science application of the voltage capture mode is for pulsar observations. Following their serendipitous discovery at 81.5 MHz by Hewish et al. (1968), much of the early research on pulsars was at low frequencies (Taylor & Manchester 1977), however the eventual quest to find more pulsars in the galactic plane, which is highly sky-noise dominated at low frequencies, and also to achieve higher precision in timing pushed the observations to higher frequencies (≳ 1 GHz). With the advent of multiple new low-frequency arrays (LOFAR) including MWA, the LOFAR (van Haarlem et al. 2013), and the long wavelength array (Taylor et al. 2012), a renaissance in low-frequency pulsar astrophysics is on the horizon. In fact, a number of low-frequency pulsar results have already started being published from these and other instruments (e.g. Bhat et al. 2014, Archibald et al. 2014, Dowell et al. 2013, Stovall et al. 2014).

While state-of-the-art pulsar backends are capable of providing far superior time resolution via phase-coherent dedispersion over large bandwidths (e.g. van Straten & Bailes 2011), the VCS functionality of the MWA is well matched to a wide range of pulsar science goals at low frequencies, particularly for long-period pulsars (spin period, P ≳ 100 ms) with dispersion measures (DMs) ≲ 200 pc cm^{− 3}, where scattering is not large enough to significantly smear the emission across the pulse period. This parameter space samples the vast majority of the local (≲ 1 kpc) pulsar population. Our 10 kHz channelisation means the effective achievable time resolution will largely be limited by the dispersive smearing within the channel, which is ~ 1 ms at DM = 100 pc cm^{− 3} at a frequency of 200 MHz. In targeted pulsar observations, this limitation is greatly reduced by coherent dedispersion. The 100 μs resolution presents an obvious challenge in observing millisecond pulsars (MSPs). It is, however, enough resolution to construct high-quality pulse profiles of pulsars with periods ⩾ that of PSRJ0437-4715 (see Bhat et al. 2014 and Figure 2). It is, in principle, possible to reconstruct the time resolution of the full bandwidth (~32.6 ns) by inverting the second PFB stage, although this is yet to be attempted.

Figure 2. Pulse profiles of four different pulsars observed with the VCS mode covering a range of DMs and periods (see Table 1 for values) to demonstrate the flexibility of the instrument. There are 64 bins, spanning the pulse period for each pulsar.

Our successful detection of PSR J0437 − 4715, a binary MSP with P = 5.75 ms and DM = 2.65 pc cm^{− 3}, also provides an excellent demonstration of the timing stability of our recording system. Multiple observations of this pulsar and other objects such as the Crab pulsar have been made over time durations of 1 h without encountering any recording glitches. A close inspection of commissioning data (Table 1) confirms that, barring some issues such as power level modulations caused by the Orbcomm satellites within the FoV (see Section 4) and abrupt changes in power levels that result from changes in the beam-former settings, the data are in general highly stable. While we are unable to accurately determine the exact instrumental offset in our timing owing to our limited data, we have verified our time stamping accuracy, to first order, by successfully combining multiple observations of PSRs J0630 − 2834 and J0534+2200 that span time intervals in the range of ~ 30 min to 20 days (Table 1). The pulse profiles from different observations align at a level suggesting that the offset is limited to ≲ 0.5 ms. We aim to further characterise this more accurately as more observations accrue over the course of time.

Table 1. Pulsars detected after processing MWA voltage data. The voltage streams were combined into incoherent beams and then processed using a PRESTO pipeline. These observations were performed at various times within the commissioning period of the VCS, hence the variety of bandwidths. PSRs J0534+2200 (the Crab) and J0528+2200 were detected within the same beam during an observation, similarly PSRs J0630-2834 and J0742-2822 within a single 31 min observation.

^aThese observations were done with only 32 tiles.

^bThe bandwidth spanned 15.36 MHz, but one of the central course channels was not recorded so only 14.08 MHz of data were summed in the dedispersed time series.

^cAll values are taken from the ATNF Pulsar catalogue http://www.atnf.csiro.au/people/pulsar/psrcat/ (Manchester et al. 2005)

Pulse broadening resulting from multipath scattering in the interstellar medium (ISM) is an important consideration at the MWA’s frequencies. Based on the galactic electron density models (NE2001; Cordes & Lazio 2002) and an observationally-established scaling relation between scattering, DM, and the observing frequency (Bhat et al. 2004), a scatter broadening $\tau _{\text{scatt}}$ ~ 1 ms is expected near the low end of the MWA band (~100 MHz) toward pulsars at distances ~ 1 kpc located within the galactic plane (b = 0°). At larger distances within the plane, scattering can be substantial, owing to the non-linear scaling of scattering with DM, with $\tau _{\text{scatt}}$ ~ 100 ms to be expected toward the galactic centre at a distance of ~ 3 kpc at ~ 100 MHz. Off the galactic plane, scattering will likely be far less severe, and is expected to be ≲ 0.3 ms at frequencies ≳ 200 MHz, thereby retaining sensitivity to the detection of most nearby pulsars, including MSPs.

As seen from Table 1 and Figure 2, our commissioning analysis so far investigated DMs up to 123 pc cm^{− 3}, and little scattering is evident at DMs ≲ 50 pc cm^{− 3} at ~ 200 MHz. Besides the Crab, known for its atypical scattering (Ellingson et al. 2013; Bhat et al. 2007), PSR J0742-2822 is the only pulsar that shows scattering in our data (Figure 2). While generally considered to be a hindrance for most pulsar studies including searches, scattering measurements provide useful means of characterising the ISM turbulence, for which the MWA frequency band is well suited.

Even with the 100 μs, 10 kHz limitations of the VCS, useful scintillation and profile studies are possible even for MSPs, as vividly demonstrated in Bhat et al. (2014). The 100-μs resolution, while not a limitation for scintillation studies where the primary goal is to investigate the time and frequency modulations of the integrated pulse emission, will likely pose a limitation in the profile studies of MSPs with P ~ a few ms. Similarly, the 10 kHz resolution may limit the scope of scintillation studies to DMs ≲ 40 pc cm^{− 3} in the MWA band. However, cyclic spectroscopy could be implemented to characterise scattering at higher DMs (Demorest 2011). Low-frequency observations can also potentially yield accurate DM measurements, provided the frequency-dependent evolution of the pulse profile is modelled in the analysis; this is important for timing-array applications such as the search for gravitational waves (e.g. Manchester et al. 2013).

3.2 Single pulse astrophysics

The last ten years has seen renewed interest in high time resolution single pulse detections, with particular enthusiasm for both rotating radio transients (RRATs; McLaughlin et al. 2006, Burke-Spolaor et al. 2011, Keane et al. 2011) and FRBs (Lorimer et al. 2007, Thornton et al. 2013, Spitler et al. 2014). Both the large FoV (hundreds or even thousands of square degrees) as well as the flexibility in observing frequency make low-frequency arrays, such as the MWA, powerful tools in the search for single pulse emission. As mentioned in Section 2.2, an incoherent sum of the elements preserves this large FoV and generates a data set that is practical to search through using modest compute resources. Once detected, the power of the interferometric array can be used to localise and study these pulses as long as the voltages have been preserved.

We have verified the ability of the MWA to observe short-duration single pulse emission using two separate observations of the Crab pulsar (PSR J0534+2200, Table 1) to observe giant pulses (Hankins et al. 2003) which, based on higher frequency observations (~5 GHz), are thought to be intrinsically ~nanosecond duration bright pulses. In the MWA band, these intrinsically short pulses are scattered to widths of tens of milliseconds (see Figure 3). We detected 51 and 47 pulses above 6 σ¹ from the observations. The brightest pulse we detected was 39 σ¹ (Figure 3), and shows a well-defined sharp peak followed by an exponential scattering tail.

Figure 3. Dedispersed (56.76 pc cm^{− 3}) total power from one of the detected Crab giant pulses observed with the VCS at 192.64 MHz. The long scattering tail (~40 ms) extends further than a single pulse period (33 ms). The median power from each fine channel was removed before dedispersion and time steps are averaged to 400 μs. Reference times on the abscissa denote seconds from the beginning of the observation.

For further information on FRBs see Trott, Tingay, & Wayth (2013) for the prospects of detecting FRBs with the MWA over a variety of potential spectral indices and Tremblay et al. (2015, in preparation) which describes the FRB pipeline being used by the MWA.

3.3 Solar science

The standard imaging modes of the MWA are being used to study the Sun (Oberoi et al. 2014) with a variety of aims, including the investigation of the quiet Sun, the detailed study of Type II and III bursts, and the characterisation of coronal mass ejections (described in Bowman et al. 2013, and references therein). However, the temporal resolution of the standard MWA imaging modes can significantly under-sample solar radio emission variability. Intense solar bursts evolve rapidly in time and frequency, but previous MWA observations show that even in a quiet state, low level radio emission from the Sun evolves strongly in both time and frequency (Oberoi et al. 2011). As such, there is a need for significantly higher time resolution observation modes with the MWA. Observations of the Sun for which voltages are captured can be used for high time resolution beamforming, either incoherent or coherent as mentioned in Section 2.2, to form high time resolution dynamic spectra (for example, as seen in Figure 4). Furthermore, the voltages can be correlated at high time resolution post-observation, in order to undertake high time resolution imaging of the Sun. Both high cadence beamforming and imaging modes are identical to those to be used for pulsars and searches for FRBs (Sections 3.1 and 3.2).

Figure 4. Dynamic spectrum spanning 5 s where the MWA’s 128 tiles have been incoherently summed from observations on MJD 56 544. Here, 12 coarse channels (half of the MWA’s typical bandwidth) were averaged to 10 ms increments and the median value of each 10 kHz fine channel was subsequently subtracted to highlight the variable emission. The vertical bands are the result of coarse channel edges where sensitivity is reduced. The narrowband, sub-second solar features highlighted here would be smoothed out with the typical integration times used in standard imaging modes. Times on the ordinate reference Unix Time.

In addition to Solar studies, the VCS is anticipated to conduct observations of interplanetary and ionospheric scintillations. High time resolution observations of satellite beacons will also be used to study the variability of ionospheric Faraday rotation.

4 INTERFERENCE ENVIRONMENT ON SUB-SECOND TIME SCALES

Using the VCS to perform a comprehensive survey of the radio frequency interference (RFI) environment at the MRO in the MWA band is anticipated, but was outside the scope of commissioning this new mode. We have, however, inevitably detected a variety of forms of interference (satellite, airborne, and ground based) throughout the commissioning process.

The most common strong source of interference we have encountered to date is generated by the Orbcomm satellite constellation, a global communications network operating in multiple bands around 137 MHz. While these transmissions to Earth are restricted to narrow bands which are, in principle, avoidable by wise coarse channel selection, the transmitted power is so much greater than astronomical sources that this additional power saturates a portion of the signal chain, with gain values typically used for observations, and power spreads out across the wider band. The MWA digitiser is presented with 80–300 MHz and always produces a complete set of 256 coarse channels, from which 24 are selected and sent to the fine PFB (see Prabu et al. 2014 for further details). Therefore, simply avoiding Orbcomm’s broadcast frequency is not sufficient to mitigate this effect. It is worth noting that incoherently summing the tiles, as we have done here, has maximum sensitivity to this (and other) interference. Rotating the phases independently to form coherent beams, and cross-correlation, both decorrelate this emission in directions other than the actual line of sight raising the overall system temperature but reducing the coherence of the interfering emission.

Orbcomm transmissions are time-domain multiplexed at 1 Hz (Figure 5), making this interference a complication for pulsar searching.

Figure 5. Total power (summed over tiles and frequencies) plot showing the effect an Orbcomm satellite has on a 184.96 MHz observation when transmitting within our beam, the envelope of the signal amplitude tracing out the tile gain pattern. These data have been averaged to 130 ms. Note the 1 Hz power variations, which would cause problems for blind periodic pulsar searches. Reference time on axis is seconds from the beginning of the observation.

The very nature of our high time resolution observations means that we are also able to detect short duration terrestrial RFI that normally goes unnoticed due to averaging, even in an RFI environment as quiet as the MRO. These detections often come about due to short-lived multi-path propagation involving the atmosphere and/or ionosphere causing temporary enhancements in communications signals (TV, FM, inter aircraft, etc.; Figure 6).

Figure 6. A dynamic spectrum from MJD 56559 spanning 0.5 s and showing short lived multi-path interference fringes from TV Channel 8 (8 MHz centred approximately around 191.5 MHz). The median from each frequency bin was subtracted before plotting. Time is averaged to 1 ms (the plot spans 0.5 s) and frequency is kept at 10 kHz. Once again, the vertical lines between coarse channels are visible through the variable emission.

The redirected RFI seen by the MWA has been shown to be a useful tool for detection of near Earth objects such as the Moon (McKinley et al. 2013) and low-orbiting space debris (Tingay et al. 2013b).

While not a broadcast form of interference, the other event which has affected our data are the step transitions generated by the MWA beamformers. Each MWA tile has an analogue beamformer, with a discreet set of ‘best value’ pointings. Instead of continuously repointing, the MWA evaluates whether or not to switch between these delay settings at fixed time intervals (typically 296 s). The combination of these quantised repointings and the large FoV of the incoherent sum leads to abrupt changes in the total power. This expresses itself as a step function in a time series in incoherently summed observations. Once again, this would not affect coherent post-processing as strongly.

5 CONCLUSIONS

In this paper, we have introduced the capability of the MWA to record its high-time (100 μs) and high frequency (10 kHz) resolution post-PFB voltage stream, the VCS. We have demonstrated the abilities of this mode by observing ten separate pulsars throughout different phases of commissioning. We have also exercised this mode on dispersed, single-pulse signals by observing the Crab pulsar and detecting roughly one giant pulse a minute with only half of the typical MWA bandwidth and only combining the tiles in an incoherent fashion. These detections, and the stability of the observations we have performed, highlight the MWA’s ability to become a workhorse in these, and other, areas of radio astronomy.