Table of contents

The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their limitations and programming difficulties. In this paper, we therefore present OpenCluster, an open-source distributed computing framework to support rapidly developing high-performance processing pipelines of astronomical big data. We first detail the OpenCluster design principles and implementations and present the APIs facilitated by the framework. We then demonstrate a case in which OpenCluster is used to resolve complex data processing problems for developing a pipeline for the Mingantu Ultrawide Spectral Radioheliograph. Finally, we present our OpenCluster performance evaluation. Overall, OpenCluster provides not only high fault tolerance and simple programming interfaces, but also a flexible means of scaling up the number of interacting entities. OpenCluster thereby provides an easily integrated distributed computing framework for quickly developing a high-performance data processing system of astronomical telescopes and for significantly reducing software development expenses.

We discuss the estimation of the interferometric visibility (fringe contrast) for the Exozodi survey conducted at the CHARA array with the JouFLU beam combiner. We investigate the use of the statistical median to estimate the uncalibrated visibility from an ensemble of fringe exposures. Under a broad range of operating conditions, numerical simulations indicate that this estimator has a smaller bias compared with other estimators. We also propose an improved method for calibrating visibilities, which not only takes into account the time interval between observations of calibrators and science targets, but also the uncertainties of the calibrators' raw visibilities. We test our methods with data corresponding to stars that do not display the exozodi phenomenon. The results of our tests show that the proposed method yields smaller biases and errors. The relative reduction in bias and error is generally modest, but can be as high as $\sim 20 \% \mbox{--}40 \%$ for the brightest stars of the CHARA data and statistically significant at the 95% confidence level (CL).

Galaxies of rare morphology are of paramount scientific interest, as they carry important information about the past, present, and future Universe. Once a rare galaxy is identified, studying it more effectively requires a set of galaxies of similar morphology, allowing generalization and statistical analysis that cannot be done when $N=1$. Databases generated by digital sky surveys can contain a very large number of galaxy images, and therefore once a rare galaxy of interest is identified it is possible that more instances of the same morphology are also present in the database. However, when a researcher identifies a certain galaxy of rare morphology in the database, it is virtually impossible to mine the database manually in the search for galaxies of similar morphology. Here we propose a computer method that can automatically search databases of galaxy images and identify galaxies that are morphologically similar to a certain user-defined query galaxy. That is, the researcher provides an image of a galaxy of interest, and the pattern recognition system automatically returns a list of galaxies that are visually similar to the target galaxy. The algorithm uses a comprehensive set of descriptors, allowing it to support different types of galaxies, and it is not limited to a finite set of known morphologies. While the list of returned galaxies is neither clean nor complete, it contains a far higher frequency of galaxies of the morphology of interest, providing a substantial reduction of the data. Such algorithms can be integrated into data management systems of autonomous digital sky surveys such as the Large Synoptic Survey Telescope (LSST), where the number of galaxies in the database is extremely large. The source code of the method is available at http://vfacstaff.ltu.edu/lshamir/downloads/udat.

Single-exposure spectra in large spectral surveys are valuable for time domain studies such as stellar variability, but there is no available method to eliminate cosmic rays for single-exposure, multi-fiber spectral images. In this paper, we describe a new method to detect and remove cosmic rays in multi-fiber spectroscopic single exposures. Through the use of two-dimensional profile fitting and a noise model that considers the position-dependent errors, we successfully detect as many as 80% of the cosmic rays and correct the cosmic ray polluted pixels to an average accuracy of 97.8%. Multiple tests and comparisons with both simulated data and real LAMOST data show that the method works properly in detection rate, false detection rate, and validity of cosmic ray correction.

Modern Astrophysics is based on multi-wavelength data organized into large and heterogeneous catalogs. Hence, the need for efficient, reliable and scalable catalog cross-matching methods plays a crucial role in the era of the petabyte scale. Furthermore, multi-band data have often very different angular resolution, requiring the highest generality of cross-matching features, mainly in terms of region shape and resolution. In this work we present C³ (Command-line Catalog Cross-match), a multi-platform application designed to efficiently cross-match massive catalogs. It is based on a multi-core parallel processing paradigm and conceived to be executed as a stand-alone command-line process or integrated within any generic data reduction/analysis pipeline, providing the maximum flexibility to the end-user, in terms of portability, parameter configuration, catalog formats, angular resolution, region shapes, coordinate units and cross-matching types. Using real data, extracted from public surveys, we discuss the cross-matching capabilities and computing time efficiency also through a direct comparison with some publicly available tools, chosen among the most used within the community, and representative of different interface paradigms. We verified that the C³ tool has excellent capabilities to perform an efficient and reliable cross-matching between large data sets. Although the elliptical cross-match and the parametric handling of angular orientation and offset are known concepts in the astrophysical context, their availability in the presented command-line tool makes C³ competitive in the context of public astronomical tools.

We describe the SpArc science gateway for spectral data obtained using the Fourier Transform Spectrometer (FTS) in operation at the Mayall 4-m telescope at the Kitt Peak National Observatory during the period from 1975 through 1995. SpArc is hosted by Indiana University Bloomington and is available for public access. The archive includes nearly 10,000 individual spectra of more than 800 different astronomical sources including stars, nebulae, galaxies, and solar system objects. We briefly describe the FTS instrument itself and summarize the conversion of the original interferograms into spectral data and the process for recovering the data into FITS files. The architecture of the archive is discussed and the process for retrieving data from the archive is introduced. Sample use cases showing typical FTS spectra are presented.

We present a new procedure for the internal (night-to-night) calibration of timeseries spectra, with specific applications to optical AGN reverberation mapping data. The traditional calibration technique assumes that the narrow [O iii] λ5007 emission-line profile is constant in time; given a reference [O iii] λ5007 line profile, nightly spectra are aligned by fitting for a wavelength shift, a flux rescaling factor, and a change in the spectroscopic resolution. We propose the following modifications to this procedure: (1) we stipulate a constant spectral resolution for the final calibrated spectra, (2) we employ a more flexible model for changes in the spectral resolution, and (3) we use a Bayesian modeling framework to assess uncertainties in the calibration. In a test case using data for MCG+08-11-011, these modifications result in a calibration precision of ∼1 millimagnitude, which is approximately a factor of five improvement over the traditional technique. At this level, other systematic issues (e.g., the nightly sensitivity functions and Feii contamination) limit the final precision of the observed light curves. We implement this procedure as a python package (mapspec), which we make available to the community.

I explore whether small or large teams produce the most important astronomical results, on average, using citation counts as our metric. I present evidence that citation counts indicate the importance of papers. For the 1343 papers published in A&A, ApJ, and MNRAS in 2012 January-February, I  considered 4.5 years worth of citations. In each journal, there are larger citation counts for papers from large teams than from small teams by a factor of about 2. To check whether the results from 2012 were unusual, I collected data from 2013 for A&A and found it to be the same as that for 2012. Could the preponderance of papers by large teams be due to self-citations (i.e., citing and cited papers sharing one or more authors)? To answer this, I looked at 136 papers with one to 266 authors and discovered a linear relation that ranges from a 12.7% self-citation rate for single-author papers to a 45.9% self-citation rate for papers with 100 authors. Correcting for these factors is not enough to explain the predominance of the papers with large teams. Then I computed citations per author. While large teams average more citations than small ones by a factor of 2, individuals on small teams average more citations than individuals on large teams by a factor of 6. The papers by large teams often have far more data, but those by small teams tend to discuss basic physical processes.

Extensive infrared spectral surveys, such as the APOGEE survey in the H-band, are now being conducted, many targeting the Galactic Bulge and recording observations of primarily red giant stars. However, because stars of different masses converge to the red giant region, the masses of single red giant stars are poorly constrained. These surveys are now using spectral resolving powers that are high enough to measure the equivalent widths of individual spectral lines, which are mostly from molecular species. Because other observations can constrain or determine the star's luminosity and radius, we have computed spherical stellar atmospheres for a fixed luminosity and radius but for a range of masses. We then computed the H-band flux spectrum for each model and searched for spectral lines that are sensitive to mass. Our synthetic spectra reveal many lines of CO that become weaker with increasing stellar mass. To explore this, we created a ratio of equivalent widths using a representative, unblended CO line and an unblended OH line that did not vary with mass. We found that this ratio varied about 30% over the mass range from $0.8\,{M}_{\odot }$ to $2.4\,{M}_{\odot }$. We repeated the spectral analysis using spherical model stellar atmospheres computed with a composition $\approx 1/3$ solar and found that the ratio displayed a very similar dependence on mass. The presence in the H-band of spectral features sensitive to the masses of red giant stars opens up the potential of constraining more tightly the physical properties of the stars making up the galactic bulge and globular clusters.

In this work, we adopted a CLEAN-based method to determine the scatter time, τ, from archived pulsar profiles under both the thin screen and uniform medium scattering models and to calculate the scatter time frequency scale index α, where $\tau \propto {\nu }^{\alpha }$. The value of α is −4.4, if a Kolmogorov spectrum of the interstellar medium turbulence is assumed. We deconvolved 1342 profiles from 347 pulsars over a broad range of frequencies and dispersion measures. In our survey, in the majority of cases the scattering effect was not significant compared to pulse profile widths. For a subset of 21 pulsars scattering at the lowest frequencies was large enough to be measured. Because reliable scatter time measurements were determined only for the lowest frequency, we were limited to using upper limits on scatter times at higher frequencies for the purpose of our scatter time frequency slope estimation. We scaled the deconvolved scatter time to 1 GHz assuming $\alpha =-4.4$ and considered our results in the context of other observations which yielded a broad relation between scatter time and dispersion measure.

A recent publication has suggested a method to determine the masses and radii of the components of an eclipsing system using only a light curve and radial velocities of one component. If true, this would have immediate impact in expediting the study of transiting extrasolar planet and brown dwarf systems. The method is intended for situations where the mass ratio is significantly different from zero, but implicitly also requires the assumption that the mass ratio is negligible. We investigate both cases, finding that when the mass ratio is significant the method is mathematically identical to existing approaches, and when the mass ratio is negligible the equations become undefined. We therefore conclude that the method cannot be used to measure the physical properties of such systems from observations alone.

We present the 3.5 years monitoring results of 225 GHz opacity at the summit of the Greenland ice sheet (Greenland Summit Camp) at an altitude of 3200 m using a tipping radiometer. We chose this site as our submillimeter telescope (Greenland Telescope) site, because conditions are expected to have low submillimeter opacity and because its location offers favorable baselines to existing submillimeter telescopes for global-scale Very Long Baseline Interferometry. The site shows a clear seasonal variation with the average opacity lower by a factor of two during winter. The 25%, 50%, and 75% quartiles of the 225 GHz opacity during the winter months of November through April are 0.046, 0.060, and 0.080, respectively. For the winter quartiles of 25% and 50%, the Greenland site is about 10%–30% worse than the Atacama Large Millimeter/submillimeter Array (ALMA) or the South Pole sites. Estimated atmospheric transmission spectra in winter season are similar to the ALMA site at lower frequencies ($\lt 450$ GHz), which are transparent enough to perform astronomical observations almost all of the winter time with opacities $\lt 0.5$, but 10%–25% higher opacities at higher frequencies ($\gt 450$ GHz) than those at the ALMA site. This is due to the lower altitude of the Greenland site and the resulting higher line wing opacity from pressure-broadened saturated water lines in addition to higher dry air continuum absorption at higher frequencies. Nevertheless, half of the winter time at the Greenland Summit Camp can be used for astronomical observations at frequencies between 450 GHz and 1000 GHz with opacities $\lt 1.2$, and 10% of the time show $\gt 10 \%$ transmittance in the THz (1035 GHz, 1350 GHz, and 1500 GHz) windows. Summer season is good for observations at frequencies lower than 380 GHz. One major advantage of the Greenland Summit Camp site in winter is that there is no diurnal variation due to the polar night condition, and therefore the durations of low-opacity conditions are significantly longer than at the ALMA site. Opacities lower than 0.05 or 0.04 can continue for more than 100 hr. Such long stable opacity conditions do not occur as often even at the South Pole; it happens only for the opacity lower than 0.05. Since the opacity variation is directly related to the sky temperature (background) variation, the Greenland Summit Camp is suitable for astronomical observations that need unusually stable sky background.

We present the prototype telescope for the Next Generation Transit Survey, which was built in the UK in 2008/2009 and tested on La Palma in the Canary Islands in 2010. The goals for the prototype system were severalfold: to determine the level of systematic noise in an NGTS-like system; demonstrate that we can perform photometry at the (sub) millimagnitude level on transit timescales across a wide-field; show that it is possible to detect transiting super-Earth and Neptune-sized exoplanets and prove the technical feasibility of the proposed planet survey. We tested the system for around 100 nights and met each of the goals above. Several key areas for improvement were highlighted during the prototyping phase. They have been subsequently addressed in the final NGTS facility, which was recently commissioned at ESO Cerro Paranal, Chile.

Hyperfine Structure Fitting (HfS) is a tool to fit the hyperfine structure of spectral lines with multiple velocity components. The HfS_nh3 procedures included in HfS simultaneously fit the hyperfine structure of the NH₃ (J, K) = (1, 1) and (2, 2) transitions, and perform a standard analysis to derive ${T}_{\mathrm{ex}}$, NH₃ column density, ${T}_{\mathrm{rot}}$, and ${T}_{{\rm{k}}}$. HfS uses a Monte Carlo approach for fitting the line parameters. Special attention is paid to the derivation of the parameter uncertainties. HfS includes procedures that make use of parallel computing for fitting spectra from a data cube.

ESASky is a science-driven discovery portal to explore the multi-wavelength sky and visualize and access multiple astronomical archive holdings. The tool is a web application that requires no prior knowledge of any of the missions involved and gives users world-wide simplified access to the highest-level science data products from multiple astronomical space-based astronomy missions plus a number of ESA source catalogs. The first public release of ESASky features interfaces for the visualization of the sky in multiple wavelengths, the visualization of query results summaries, and the visualization of observations and catalog sources for single and multiple targets. This paper describes these features within ESASky, developed to address use cases from the scientific community. The decisions regarding the visualization of large amounts of data and the technologies used were made to maximize the responsiveness of the application and to keep the tool as useful and intuitive as possible.

Astronomical data does not always use Cartesian coordinates. Both all-sky observational data and simulations of rotationally symmetric systems, such as accretion and protoplanetary disks, may use spherical polar or other coordinate systems. Standard displays rely on Cartesian coordinates, but converting non-Cartesian data into Cartesian format causes distortion of the data and loss of detail. Here, I  demonstrate a method using standard techniques from computer graphics that avoids these problems with three-dimensional data in arbitrary coordinate systems. The method adds minimum computational cost to the display process and is suitable for both realtime, interactive content, and producing fixed rendered images and videos. Proof-of-concept code is provided which works for data in spherical polar coordinates.