Knowledgebase: Table Upload
My multi-object table search does not work! Why?
The following page describes the basic IPAC Table Format, which can be used with IRSA Services that support multi-object searches.
Please pay particular attention to the "DBMS Constraints" links, which provide additional rules that are imposed by IRSA Services.
Can I input a list of object names to search for?
You cannot input a list of object names directly into the Catalog Search Tool. However, you can feed your list (with no header) into the IPAC Table Reformat and Validation Service:
http://irsa.ipac.caltech.edu/applications/TblCheck/
If you click "Apply name resolution/coordinate transformation", the tool will check SIMBAD and NED to determine the coordinates. These will then be output, and can be fed into the Catalog Search Tool.
When I try to upload a table, I get an error message saying IRSA could not recognize this table format. What does that mean?
This usually means that the table you uploaded is in a binary file format, such as .doc, .rtf or .xls.
In order for a table to be read by IRSA's services, it must be in a plain text ASCII file. If you are using Microsoft Word, be aware documents may appear to be in plain text, but can still contain hidden formatting if saved in any file format other than plain text (.txt).
For the best results, create the file in a plain text editor such as vi, emacs, Notepad in Windows, or TextEdit in Mac. If your data is in a Microsoft Excel spreadsheet, you can save the file as a .csv file.
When I upload a table, I get an error message saying that records in the table do not contain uniform number of columns. What does that mean?
This error indicates that the table you uploaded contains too many data columns and too few column names.
If you are using IPAC ASCII Column-Aligned, or tab- or comma-delimited formats, the number of cells containing data must not exceed the number of column names.
In the following example, there are three named columns: A, B, and C. However, there is a row with four cells of data:
A,B,C 5,2, 7,6,9,11
In the above example, the "11" is orphan data because it is not aligned with column A, B, or C.
If you are using IPAC ASCII Column-Aligned format, placing data under the vertical bars in the header will result in both this error message and a "failed to parse" error message.
Add additional column names in the header row to identify each data cell, and make sure that data entries are properly aligned with the correct header columns.
When I upload a table, I get an error message saying "We cannot identify columns to use as coordinates or a column to use as object names/locations." What should I do?
This error message means that the column names in your table do not sufficiently describe an object name, or the locations on the sky using ra, dec, glon, glat, etc.
Clean up the column names so they are recognizable by IRSA services. See Supported Coordinate Systems for more information.
Example:
| jcgRA | jcgDec | size | 150.3814 2.3606 60.0 150.2794 2.1560 60.0 149.8873 2.0789 60.0 150.2323 1.9599 60.0 150.5407 2.5196 60.0 149.9343 2.4426 60.0
In the above example, there is extraneous text appended to RA and Dec, rendering the column name unrecognizable.
When uploading a table, I get an error saying "Failed to parse the table." What should I do?
Three common conditions may result in this error message.
Reason #1: When using IPAC Table Format, data are located beneath the vertical bars in the header.
Solution: Align the data with the columns in the header. (See IPAC Table Format)
Reason #2: Space is used as data delimiter, but also exists within the data, making it uninterpretable.
Solution: If you are using the tab-delimited format, carefully check your data for extra tabs that the system is interpreting as new columns. Be aware that tabs are not visually obvious when viewed on screen or in print; it's best to use a text editor that will allow you to view the file's underlying formatting. (See Best Practices For Successful Table Upload)
Reason: #3 Table contains non-printable ASCII characters.
A non-printable ASCII character in a table, such as ^[ (escape) or ^\\ (file separator), can create problems during the upload process. Tabs are one of the most common causes of failed uploads because they are considered characters, even if they don't appear on print-outs, or even on-screen. When used in a table, the tab character is translated into its ASCII counterpart (^I), which is not interpretable during table upload.
Solution: Remove the characters from the file. Cleaning up tabs is done differently, based on your operating system and text editor:
- On Max OSX, Linux and Sun OS, use the Unix command cat to view a file's non-printable characters:
cat -tev filename
Here are two examples:
Example 1: cat -tev ipac_good_table.tbl
| lon | lat |$ | double | double |$ 237.140482 -28.797019 $ 237.3 -28.0 $
Example 2: cat -tev tab_delim_table.txt
object^Ira^Idec$ M56^I289.147941100^I30.184500500 $ ic4710^I277.158208330^I-66.982277780 $ hoix^I149.383333330^I69.045833330 $
This will bring your file to the screen with all hidden characters, such as tabs (^I) and end of lines ( $ ) indicated. Tabs may seem like spaces and can easily be found using this command, and fixed. Additional blank space at the end of each line causes parser errors; if the end of line ( $ ) characters do not line up, then delete the blank space until they do. - In Microsoft Word on Windows machines, view all tabs in a document by toggling on formatting marks (click the toolbar button that looks like a paragraph symbol, similar to the pi symbol). Remove the arrows embedded within the table, and then save the file as "text only." Here is a screenshot example:
When uploading a table, the search results return objects at coordinates substantially different than the intended input coordinates.
Reason: The table is using IPAC ASCII Column-Aligned format, and the columns are misaligned.
Solution: Make sure the table data are contained within the boundaries set by the vertical lines in the column header rows.
When I upload a table, I get the error "Semi-major axis length column named 'major' or 'radius' not found in input table..." What should I do?
This error indicates that you have not entered a search radius. Please try again with a valid search radius.
When I upload a table in IPAC Table Format, I get the following error: "Unexpected [(] in column name 'dist(mpc)_01'". Why?
In IPAC Table Format, column names can be the combination of characters, numbers, or underscores ("_"). Blanks, white spaces, tabs, dashes ("-"), or any other special characters are not allowed as part of column names. In this case, the parentheses must be removed.
When I upload a table into the Catalog Search Tool, I get the error "Value exceeds limit of INTEGER precision". What should I do?
Your table probably contains a column with data type "int", but with values that have more digits than allowed by this data type. Try changing "int" to "long".
How can I make my very large catalog query run faster?
Here are some tips for making your large catalog search return faster:
1. If you suspect that your query might return results larger than 2 GB, you will need to break it up into smaller queries. It can be difficult to predict the size of the returned results. It depends on the constraints you are interested in, the number of columns you want, and the density of objects on the sky. You can do a test on a 1 sq deg patch of the sky, for instance, and scale up until the result is just under 2 GB.
2. Query on indexed columns as much as possible. If you are using the web interface, you can see which columns are indexed in the table that allows you to select which columns to download. The indexed columns are marked with an X. For the AllWISE catalog, the indexed columns are dec and glat.
3. Polygon searches do take advantage of the spatial indexing. If users have spatial constraints, they can be entered as polygon limits or entered into the column constraint table.