proposal: github engine support options

[junnplus]

Since the github engine is supported. I found it hard to use.

usage

for now, we can take a tour as below:

we need to export our Github Access Token in the environment variable to get more free usage in github api

$ export GITHUB_TOKEN=your_token; make run

create github-engine based database

mysql> create database datafuselabs engine=github;
Query OK, 0 rows affected (2.01 sec)
Read 0 rows, 0 B in 2.003 sec., 0 rows/sec., 0 B/sec.

It requires environment variables to be set when the service starts. It's very unfriendly to users. ):
We can consider supporting users' tokens through engine options, like:

mysql> create database datafuselabs engine=github(token="xxx");

This has other benefits:

• Support github enterprise repo.
• Specified table type, like only pull pr.
• Specified repo or filter some repo.
• and so on.

github as a table engine is also difficult to use. https://github.com/datafuselabs/databend/blob/main/query/src/storages/storage_factory.rs#L72-L89
I might just want to create a table to analyze the repo.

mysql> create table databend engine = GITHUB_REPO_INFO_TABLE_ENGINE;
Query OK, 0 rows affected (0.06 sec)
Read 0 rows, 0 B in 0.064 sec., 0 rows/sec., 0 B/sec.

mysql> select * from databend;
ERROR 1105 (HY000): Code: 54, displayText = Github table need owner in its mata.

We can consider a unified way of using, like:

create table databend_prs engine=github(token="xxx", type="pr");

[PsiACE]

I might just want to create a table to analyze the repo.

Yes. We need a way to identify and organise the information we want.

[BohuTANG]

Good idea, thank you @junnplus

[BohuTANG]

github datasource is a database engine, we should create it as:

create database datafuselabs engine=github; -- datafuselabs must be your org name

[BohuTANG]

Proposal emitted: #3431