GAIA: new simplified cross match method #3320

cosmoJFH · 2025-05-16T16:15:05Z

Dear astroquery team,

The current implementation of the astroquery.gaia built-in cross-match tool relies in a sub-optimal intermediate ADQL query. Furthermore (and after uploading a table to the user space), it requires a 3-step approach involving different functionalities (each of them requiring multiple "user clicks"):

Update the user table metadata to flag the positional (ra & dec) fields using a dedicated method ("update_user_table" - see Sect 2.5 in the astroquery.gaia documentation)
Launch the built-in cross-match method ("cross_match" - see Sect 2.6 in the astroquery.gaia documentation)
Launch a sub-optimal and complicated ADQL query using the "launch_job" or "launch_job_async" method (see Sect 2.6 in the astroquery.gaia documentation).

We propose to streamline this procedure to improve the user experience and hopefully promote the usage of this functionality by updating the current implementation.

The proposed mechanism:

allows to obtain a basic cross-match using a single step (versus three steps currently).
allows to perform cross-matches between: user_table X user_table, user_table X GACS_table, and GACS_table X GACS_table (as in the current implementation).

We have developed the new method cross_match_basic as well as new tests. This change is backward compatible with the present implementation of the cross-match method.

cc @esdc-esac-esa-int

jira: GAIASWRQ-25

codecov · 2025-05-16T17:37:38Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.97%. Comparing base (823e49f) to head (e63ae6b).
Report is 16 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3320      +/-   ##
==========================================
+ Coverage   69.88%   69.97%   +0.08%     
==========================================
  Files         232      232              
  Lines       19761    19814      +53     
==========================================
+ Hits        13810    13864      +54     
+ Misses       5951     5950       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bsipocz

Thanks! Overall looks good, I just have minor comments.

And please rebase so it picks up the recent fixes and will pass the RTD build.

bsipocz · 2025-06-03T04:58:37Z

astroquery/gaia/core.py

@@ -853,15 +848,155 @@ def load_user(self, user_id, *, verbose=False):

        return self.is_valid_user(user_id=user_id, verbose=verbose)

+    def cross_match_basic(self, *, table_a_full_qualified_name, table_a_column_ra, table_a_column_dec,


I didn't notice this weirdness in the argument names before, e.g. why do we need qualified in there, what does it even mean?

Anyway, I see that it's some historical heritage that we haven't spotted before, so let's keep it to have a consistent API.

"qualified" means that the name of the table must contain the schema. Something like "my_schema.my_table"

bsipocz · 2025-06-03T05:01:18Z

astroquery/gaia/core.py

+            #. updates the user table metadata to flag the positional RA/Dec columns;
+            #. launches a positional cross-match as an asynchronous query;
+            #. returns all the columns from both tables plus the angular distance (deg) for the cross-matched sources.


Unfortunately this won't be auto resolved into numbers in the docstrings, only when sphinx rendered, so could you please instead spell out the numberings?

Done. Changes committed to the branch.

bsipocz · 2025-06-03T05:02:32Z

astroquery/gaia/core.py

+            the ‘dec’ column in the table table_b_full_qualified_name
+        results_name : str, optional, default None
+            custom name defined by the user for the job that is going to be created
+        radius : float (arc. seconds), optional, default 1.0


I think it would be useful to accept Quantity radius, too, not just floats

The methods cross_match and cross_match_basic now accept radius as a Quantity

…tream

…us as a Quantiy

cosmoJFH · 2025-06-03T09:39:15Z

Branch rebased.

bsipocz · 2025-06-03T14:37:38Z

astroquery/gaia/core.py

+        step:
+            1. updates the user table metadata to flag the positional RA/Dec columns;


I suspect this lack of an empty line is the cause for the sphinx failure

Suggested change

step:

1. updates the user table metadata to flag the positional RA/Dec columns;

step:

1. updates the user table metadata to flag the positional RA/Dec columns;

Done. Changes committed to the branch.

Thanks! I'll investigate what goes wrong with the oldest and dev builds, I'm certain it is unrelated, yet it's something new and concerning so I rather check them out first to be sure.

bsipocz

Thanks!

CI failures are unrelated and have been fixed in main, thus I go ahead with the merge as is.

bsipocz added this to the v0.4.11 milestone May 16, 2025

bsipocz added gaia utils.tap labels May 16, 2025

cosmoJFH force-pushed the ESA_gaia_GAIASWRQ-25_improve_cross-match branch 3 times, most recently from a80c138 to dc0fac9 Compare May 16, 2025 17:32

cosmoJFH force-pushed the ESA_gaia_GAIASWRQ-25_improve_cross-match branch 9 times, most recently from 49dbbfe to 34b324d Compare May 19, 2025 09:24

cosmoJFH force-pushed the ESA_gaia_GAIASWRQ-25_improve_cross-match branch from 7fb5b9d to e63ae6b Compare June 2, 2025 09:43

bsipocz reviewed Jun 3, 2025

View reviewed changes

Jorge Fernandez Hernandez added 12 commits June 3, 2025 11:37

GAIASWRQ-25 new x-match method.

3b373bd

GAIASWRQ-25 Fix issues in the documentation

3ff45d0

GAIASWRQ-25 Fix issues in the method __update_ra_dec_columns

a2967de

GAIASWRQ-25 Include PR reference

ed1b890

GAIASWRQ-25 Fix E126 continuation line over-indented for hanging indent

80ea8de

GAIASWRQ-25 Fix tests due to the last changes in method cross_match_s…

5cd6dd9

…tream

GAIASWRQ-25 Increase coverage

605efd9

GAIASWRQ-25 Update documentation for section 2.6

66ebc71

GAIASWRQ-25 Rename method to cross_match_basic

7151b9f

GAIASWRQ-25 make q3c indexes for ra and dec

01db07e

GAIASWRQ-25 Include extra comment at the end for section 2.6

5f7adff

GAIASWRQ-25 The methods cross_match and cross_match_basic accept radi…

8a7e62f

…us as a Quantiy

cosmoJFH force-pushed the ESA_gaia_GAIASWRQ-25_improve_cross-match branch from 09f2f1b to f7948be Compare June 3, 2025 09:38

bsipocz reviewed Jun 3, 2025

View reviewed changes

GAIASWRQ-25 spell out the numberings

2676412

cosmoJFH force-pushed the ESA_gaia_GAIASWRQ-25_improve_cross-match branch from c05573e to 2676412 Compare June 3, 2025 14:50

bsipocz approved these changes Jun 4, 2025

View reviewed changes

bsipocz merged commit 8b57bef into astropy:main Jun 4, 2025
8 of 10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

GAIA: new simplified cross match method #3320

GAIA: new simplified cross match method #3320

Uh oh!

cosmoJFH commented May 16, 2025 •

edited

Loading

Uh oh!

codecov bot commented May 16, 2025 •

edited

Loading

Uh oh!

bsipocz left a comment

Uh oh!

bsipocz Jun 3, 2025

Uh oh!

cosmoJFH Jun 3, 2025

Uh oh!

bsipocz Jun 3, 2025

Uh oh!

cosmoJFH Jun 3, 2025

Uh oh!

bsipocz Jun 3, 2025

Uh oh!

cosmoJFH Jun 3, 2025

Uh oh!

cosmoJFH commented Jun 3, 2025

Uh oh!

bsipocz Jun 3, 2025

Uh oh!

cosmoJFH Jun 3, 2025

Uh oh!

bsipocz Jun 3, 2025

Uh oh!

bsipocz left a comment

Uh oh!

Uh oh!

Uh oh!

		@@ -853,15 +848,155 @@ def load_user(self, user_id, *, verbose=False):

		return self.is_valid_user(user_id=user_id, verbose=verbose)

		def cross_match_basic(self, *, table_a_full_qualified_name, table_a_column_ra, table_a_column_dec,

		step:
		1. updates the user table metadata to flag the positional RA/Dec columns;

Uh oh!

GAIA: new simplified cross match method #3320

GAIA: new simplified cross match method #3320

Uh oh!

Conversation

cosmoJFH commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cosmoJFH commented Jun 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cosmoJFH commented May 16, 2025 •

edited

Loading

codecov bot commented May 16, 2025 •

edited

Loading