Feat: add v2fly domain-list-community (#13)

* feat: add v2fly domain-list-community as source

* docs: add v2fly in sources section

Author: bootmortis

README.fa.md

@@ -164,11 +164,12 @@ rule-providers:
 - **qv2ray_schema.json:** فایل قابل استفاده در کلاینت [Qv2ray](https://github.com/Qv2ray/Qv2ray).
 - **shadowrocket.conf:** فایل قابل استفاده در کلاینت [Shadowrocket](https://apps.apple.com/us/app/shadowrocket/id932747118).
 
-## منابع
+## منابع و قدردانی‌ها
 
 - دامنه‌های ایران:
   - [سازمان فناوری اطلاعات ایران](https://g2b.ito.gov.ir/index.php/site/list_ip)
   - [سامانه مدیریت اینترنت مشتریان شرکت مخابرات ایران](https://adsl.tci.ir/panel/sites)
+  - [انجمن لیست دامنه v2fly](https://github.com/v2fly/domain-list-community)
   - [لیست شخصی][link-custom]
 - تبلیغات:
   - [PersianBlocker](https://github.com/MasterKia/PersianBlocker) (لایسنس AGPL-3.0)

README.md

@@ -181,11 +181,12 @@ go run ./ --outputdir=../
 - **qv2ray_schema.json:** Importable json schema that can be used in [Qv2ray](https://github.com/Qv2ray/Qv2ray).
 - **shadowrocket.conf:** Importable conf file that can be used in [Shadowrocket](https://apps.apple.com/us/app/shadowrocket/id932747118).
 
-## Source
+## Sources & Acknowledgements
 
 - Iran Domains:
   - [ITO GOV](https://eservices.ito.gov.ir/page/iplist) - [Mirror](https://github.com/bootmortis/ito-gov-mirror)
   - [ADSL TCI](https://adsl.tci.ir/panel/sites)
+  - [V2fly Domain List Community](https://github.com/v2fly/domain-list-community)
   - [Custom List][link-custom]
 - ADs:
   - [PersianBlocker](https://github.com/MasterKia/PersianBlocker) (AGPL-3.0 License)

src/constants.py

@@ -1,6 +1,7 @@
 # https://eservices.ito.gov.ir/page/iplist
 g2b_gov_url = "https://raw.githubusercontent.com/bootmortis/ito-gov-mirror/main/out/domains.csv"
 ads_url = "https://raw.githubusercontent.com/MasterKia/PersianBlocker/main/PersianBlockerHosts.txt"
+v2fly_base_url = "https://raw.githubusercontent.com/v2fly/domain-list-community/master/data/"
 
 # input files
 adsl_tci_file_path = "src/data/adsl_tci.txt"

src/get_domians.py

@@ -33,3 +33,20 @@ def ads() -> Iterable[str]:
     ads = filter(utils.is_not_ip, ads)
     ads = filter(utils.is_url, ads)
     return sorted(ads)
+
+def v2fly(filename = "category-ir") -> Iterable[str]:
+    resp = requests.get(consts.v2fly_base_url + filename)
+    resp.raise_for_status()
+
+    domains = []
+    for line in resp.text.splitlines():
+        line = re.sub(r'(?m)^\s*#.*\n?', '', line)
+        if line.startswith("include:"):
+            domains.extend(v2fly(line.split(":")[1]))
+        else:
+            domains.append(line)
+    
+    domains = (domain.strip() for domain in domains if domain.strip() != '')
+    domains = filter(utils.is_not_ip, domains)
+    domains = filter(utils.is_url, domains)
+    return sorted(domains)

src/main.py

@@ -44,6 +44,7 @@ def collect_and_clean_domains(*domain_set: Iterable[Iterable[str]]) -> Iterable[
     all_domains = collect_and_clean_domains(
         get_domians.g2b_ito_gov(),
         get_domians.adsl_tci(),
+        get_domians.v2fly(),
     )
 
     # Divide info