Cloud Data Access
IRSA datasets are available on Amazon Web Services (AWS) S3 cloud storage under a Space Act Agreement between NASA and AWS. The files are hosted through the AWS Open Data Sponsorship Program, allowing them to be downloaded or accessed directly without credentials or egress charges.
Accessing IRSA Data on AWS S3
Here are some key terms for accessing data on S3:
- Bucket: Container holding objects.
- Object: Analogous to a file. We ignore the minor differences here and just use "file" for simplicity.
- S3 prefix: Path to a directory, relative to the bucket.
- S3 key: Path to a file, relative to the bucket.
There are many ways to access the buckets, including:
- Python: See our Notebook Tutorials for python demonstrations.
- Command line: See the AWS Command Line Interface documentation.
- Basic example:
aws s3 ls --no-sign-request "s3://${bucket_name}/${prefix}/"
- Basic example:
- Bulk downloads: Download scripts for catalogs can be found through the links at Catalog Bulk Download.
SPHEREx
Bucket name: nasa-irsa-spherex
Bucket region: us-east-1
Quick Release (QR)
AWS Open Data Registry page: https://registry.opendata.aws/spherex-qr/
Browsable directories: https://nasa-irsa-spherex.s3.us-east-1.amazonaws.com/index.html
Data products:
See SPHEREx Data Products Available at IRSA for more information about the following products and the file naming conventions.
Spectral Images (FITS)
Spectral Images (extension 1), per-pixel status and processing flags (2), variance map (3), zodiacal model (4),
exposure-averaged PSF (5), and wavelength WCS (6). Organized by planning period.
- S3 prefix: qr/level2
Absolute Gain Matrix (FITS)
Pixel-to-pixel gain variations within a single spectral channel and relative gain differences across channels.
- S3 prefix: qr/abs_gain_matrix
Exposure-Averaged PSF (FITS)
Wavelength-dependent point spread function (PSF) estimates on a fine positional grid across each detector.
- S3 prefix: qr/average_psf
Dark Current (FITS)
Per pixel dark current.
- S3 prefix: qr/dark
Dichroic (FITS)
Map of pixels affected by flux attenuation due to the dichroic filter.
- S3 prefix: qr/dichroic
Gain Factors (YAML)
Gain factors for each detector.
- S3 prefix: qr/gain_factors
Non-functional Pixel Map (FITS)
Map of permanently non-functioning pixels.
- S3 prefix: qr/nonfunc
Non-Linearity Correction (FITS)
Corrections applied to compensate for detector non-linearity due to gain degradation.
- S3 prefix: qr/nonlinear_pars
Readout Noise (FITS)
Per-detector read noise maps.
- S3 prefix: qr/readnoise_pars
Spectral WCS Map (FITS)
Detailed World Coordinate System (WCS) map.
- S3 prefix: qr/spectral_wcs
Euclid
Quick Release 1 (Q1)
Bucket name: nasa-irsa-euclid-q1
Bucket region: us-east-1
AWS Open Data Registry page: https://registry.opendata.aws/euclid-q1/
Browsable directories: https://nasa-irsa-euclid-q1.s3.us-east-1.amazonaws.com/index.html
Data products:
MER (FITS)
Level 2 multiwavelength space- and ground-based mosaics, organized by Tile ID.
- S3 prefix: q1/MER
MER_SEG (FITS)
Segmentation maps of detections made on the MER mosaics, organized by Tile ID.
- S3 prefix: q1/MER_SEG
NIR (FITS)
Level 2 YJH NISP calibrated frames, organized by Observation ID.
- S3 prefix: q1/NIR
VIS (FITS)
Level 2 VIS calibrated frames, organized by Observation ID.
- S3 prefix: q1/VIS
SIR (FITS)
2D spectroscopic science frames and 1D spectra, organized by Observation ID.
- S3 prefix: q1/SIR
VMPZ (FITS)
Level 3 visibility masks.
- S3 prefix: q1/VMPZ
RAW (FITS)
Level 1 VIS and NISP raw frames, organized by Observation ID.
- S3 prefix: q1/RAW
Catalogs (FITS)
MER final catalog, photometric redshift catalogs, spectroscopic catalogs organized by Tile ID.
NIR and VIS catalogs of calibrated frame detections, organized by Observation ID.
- S3 prefix: q1/catalogs
Enhanced Contributed Data Products:
Merged Catalogs (Parquet; HATS Collection)
MER final catalog, photometric redshift catalogs, and spectroscopic catalogs joined on MER Object ID into a
single Parquet dataset (HATS Catalog).
- S3 prefix: contributed/q1/merged_objects/hats
WISE
Bucket name: nasa-irsa-wise
Bucket region: us-west-2
AllWISE
AWS Open Data Registry page: https://registry.opendata.aws/wise-allwise/
Data products:
Images Atlas (FITS)
For file naming conventions, see AllWISE IBE docs and Map IBE to cloud paths.
- S3 prefix: wise/allwise/images/p3am_cdd
Enhanced Contributed Data Products:
Source Catalog (Parquet; HEALPix partitioned)
- S3 prefix: wise/allwise/catalogs/p3as_psd/healpix_k5
- Parquet name: wise-allwise.parquet
All-Sky
AWS Open Data Registry page: https://registry.opendata.aws/wise-allsky/
Data products:
Single-exposure Image Sets (FITS)
For file naming conventions, see All-Sky IBE docs and Map IBE to cloud paths.
- S3 prefix: wise/allsky/images/4band_p1bm_frm
3-Band Cryo
AWS Open Data Registry page: https://registry.opendata.aws/wise-cryo-3band/
Data products:
Single-exposure Image Sets (FITS)
For file naming conventions, see 3-Band Cryo IBE docs and Map IBE to cloud paths.
- S3 prefix: wise/cryo-3band/images/3band_p1bm_frm
Post-Cryo
AWS Open Data Registry page: https://registry.opendata.aws/wise-postcryo/
Data products:
Single-exposure Image Sets (FITS)
For file naming conventions, see Post-Cryo IBE docs and Map IBE to cloud paths.
- S3 prefix: wise/postcryo/images/2band_p1bm_frm
NEOWISE Reactivation
Bucket name: nasa-irsa-wise
Bucket region: us-west-2
NEOWISE-R
AWS Open Data Registry page: https://registry.opendata.aws/wise-neowiser/
Data products:
Single-exposure Image Sets (FITS)
For file naming conventions, see NEOWISE-R IBE docs and Map IBE to cloud paths.
- S3 prefix: wise/neowiser/images/p1bm_frm
Enhanced Contributed Data Products:
Single-exposure Source Table (Parquet; HEALPix partitioned)
Composed of yearly data releases plus an addendum, stored separately.
Replace "{PART}" below with either "yearN" (where N is an integer, 1-11) or "addendum".
- S3 prefix: wise/neowiser/catalogs/p1bs_psd/healpix_k5/{PART}
- Parquet name: neowiser-healpix_k5-{PART}.parquet
Spitzer
Bucket name: nasa-irsa-spitzer
Bucket region: us-west-2
Spitzer Enhanced Imaging Products (SEIP)
AWS Open Data Registry page: https://registry.opendata.aws/spitzer-seip
Data products:
Super Mosaics (FITS)
- S3 prefix: spitzer/seip/seip_science/images
Zwicky Transient Facility (ZTF)
Bucket name: ipac-irsa-ztf
Bucket region: us-east-1
AWS Open Data Registry page: https://registry.opendata.aws/ztf/
Data Release 23 (DR23)
Enhanced Contributed Data Products:
Objects Table (Parquet; HATS Collection)
- S3 prefix: contributed/dr23/objects/hats
Lightcurves (Parquet; HATS Collection)
- S3 prefix: contributed/dr23/lc/hats
unWISE
Bucket name: nasa-irsa-wise
Bucket region: us-west-2
AWS Open Data Registry page: https://registry.opendata.aws/wise-unwise/
Enhanced Contributed Data Products:
Time-Domain Catalog (Parquet; HEALPix partitioned)
- S3 prefix: unwise/neo7/catalogs/time_domain/healpix_k5
- Parquet name: unwise-neo7-time_domain-healpix_k5.parquet
OpenUniverse 2024 Matched Rubin and Roman Simulations
Bucket name: nasa-irsa-simulations
Bucket region: us-east-1
AWS Open Data Registry page: https://registry.opendata.aws/openuniverse2024/
Preview
Data products:
Preview of Roman simulated data products (FITS, Parquet)
- S3 prefix: openuniverse2024/roman/preview
Preview of Rubin simulated data products (FITS, Parquet)
- S3 prefix: openuniverse2024/rubin/preview
Full area release
Data products:
Full area release of Roman simulated data products (FITS, Parquet)
- S3 prefix: openuniverse2024/roman/full
Full area release of Rubin simulated data products (FITS, Parquet)
- S3 prefix: openuniverse2024/rubin/full
Requesting additional datasets: If there are other IRSA holdings that you would like us to make available in the cloud, please contact IRSA's Help Desk.