SQLAlchemy vs Glue Crawler (for SQL Server)

[matt-heery]

Recently, in the EM AWS environment, I whipped up a test database in our dev RDS database which runs SQL Server. I created a database with a number of different types, that I both used SQLALchemyConverter to transform to Metadata type, and then used GlueConverter to create a number of glue tables in the catalog. All of this was running in a lambda, and runs very quickly.

I also, ran a glue crawler on the database, which took 2 mins (ie longer and more expensive), and pulled the output of the database. I summarised the differences in this google sheet (with the original input for each column pulled from parquet).

Mainly posted this here to get people's thoughts on whether we need to:
A) Fix SQLAlchemyConverter, maybe specifically for SQLServer to align better here (if that is a good option)
B) Remove SQLAlchemyConverter from mojap-metadata if we can now use Glue crawlers for the same result
C) Do nothing (also valid)