Merge branch 'develop'

Author: jamiesun

README.md

@@ -18,6 +18,12 @@ TOUGHRADIUS uses advanced Golang technology to develop the system core, providin
 
 [quick Start](https://github.com/talkincode/toughradius/wiki/quickstart)
 
+## QABOT
+
+ToughRadius provides an AI-based technical service bot that we hope can help you solve some technical problems.
+
+[QABOT Link](https://qa.toughradius.net/)
+
 ## System structure
 
 ![architecture](assets/architecture.png)

assets/buildinfo.txt

@@ -1,8 +1,8 @@
-BuildVersion=latest v8.0.4 2023-12-31 03:36:45
+BuildVersion=latest v8.0.4 2024-01-02 02:00:12
 ReleaseVersion=v8.0.4
-BuildTime=2023-12-31 03:36:45
+BuildTime=2024-01-02 02:00:12
 BuildName=toughradius
-CommitID=f871aea6d9c0c3eda4f50423d44c241822bf3154
-CommitDate=Sun, 31 Dec 2023 03:11:03 +0800
+CommitID=375ca52501466fefb7007faa08b34611472d4e0a
+CommitDate=Tue, 2 Jan 2024 02:00:07 +0800
 CommitUser=jamiesun.net@gmail.com
-CommitSubject=2023-12-31 03:10:51 : mschapv2 auth
+CommitSubject=aibot add

docs/documents/Explanation of nouns .md

@@ -0,0 +1,87 @@
+
+## Node
+
+The management node is a technology solution used to logically group network devices for unified management and control. By grouping network devices, the administrator can more easily manage the network devices and also facilitate maintenance and management of the network devices.
+
+## CPE 
+
+CPE (Customer Premises Equipment) refers to network hardware located at the customer's location, such as a residence or business, used to provide a connection to the service provider's network. These devices include modems, routers, switches and other network equipment used to provide internet and other communication services to customers. In TeamsACS, CPE functions as the TR069 protocol client and communicates with TeamsACS.
+
+Mikrotik's RouterOS router, produced by Mikrotik, is a good example and a commonly used CPE device that provides Internet and other communication services. By communicating with TeamsACS through the TR069 protocol, administrators can manage and control CPE devices more easily.
+
+## TR069 Config 
+
+In TeamsACS, TR069 Config is mainly Configuration data of Configuration Change (3 Vendor Configuration File) type, which is sent to CPE through 'Download RPC method'
+
+Mikrotik devices are more flexible in dealing with Configuration Change (3 Vendor Configuration File) configuration data. The configuration it receives is the RouterOS script, through which more operations can be performed, such as configuring the router, setting up the firewall, etc. Mikrotik devices are a good choice to use because of their flexibility.
+
+The following is a description quoting the official [Mikrotik Wiki](https://help.mikrotik.com/docs/display/ROS/TR-069)
+
+`Configuration Change (3 Vendor Configuration File)`
+
+> The same Download RPC can be used to perform complete configuration overwrite (as intended by standard) OR configuration alteration (when URL's filename extension is ".alter").
+
+`Alter configuration`
+
+> RouterOS has a lot of configuration attributes and not everything can be ported to CWMP Parameters, that's why RouterOS provides a possibility to execute its powerful scripting language to configure any attribute. A configuration alteration (which is really a regular script execution) can be performed using Download RPC FileType="3 Vendor Configuration File" with downloadable file extension ".alter". This powerful feature can be used to configure any ROS attributes which are not available through CWMP Parameters.
+
+`Overwrite all configurations`
+
+> Full ROS configuration overwrite can be performed using Download RPC FileType="3 Vendor Configuration File" with any URL file name (except with ".alter" extension).
+
+### TR069 config session
+
+The TR069 config can be triggered manually at any time in the TeamsACS system. A full execution log is recorded for each release so that the user can view the status of the release at any time. This increases the transparency and traceability of the system and allows managers to monitor and manage the issuance of TR069 configs more effectively.
+
+
+## Firmware config
+
+In TeamsACS, Firmware config is mainly configuration data of type (1 Firmware Upgrade Image), which is sent to CPE through "download RPC method"
+
+The following is a description quoting the official [Mikrotik Wiki](https://help.mikrotik.com/docs/display/ROS/TR-069)
+
+
+`RouterOS Update (1 Firmware Upgrade Image)`
+
+> CWMP standard defines that CPE's firmware can be updated using Download RPC with FileType="1 Firmware Upgrade Image" and single URL of a downloadable file (HTTP and HTTPS are supported). Standard also states that downloaded file can be any type and vendor specific process can be applied to finish firmware update. Because MikroTik's update is package based (and also for extra flexibility), an XML file is used to describe firmware upgrade/downgrade. For now, XML configuration supports providing multiple URLs of files, which will be downloaded and applied similarly as regular RouterOS update through firmware/package file upload.
+
+> An example of RouterOS bundle package and tr069-client package update (don't forget to also update tr069-client package). An XML file should be put on some HTTP server, which is accessible from CPE for download. Also, downloadable RouterOS package files should be accessible the same way (can be on any HTTP server). Using ACS execute Download RPC with URL pointing to XML file (e.g. "https://example.com/path/upgrade.xml") with contents:
+
+```
+<upgrade version="1" type="links">
+   <config/>
+   <links>
+       <link>
+          <url>https://example.com/routeros-mipsbe-X.Y.Z.npk</url>
+       </link>
+       <link>
+          <url>https://example.com/tr069-client-X.Y.Z-mipsbe.npk</url>
+       </link>
+   </links>
+</upgrade>
+```
+
+> CPE will download XML, parse/validate its contents, download files from provided URLs and try to upgrade. The result will be reported with TransferComplete RPC.
+
+
+## Factoryreset config
+
+
+The following is a description quoting the official [Mikrotik Wiki](https://help.mikrotik.com/docs/display/ROS/TR-069)
+
+
+`RouterOS default configuration change (X MIKROTIK Factory Configuration File)`
+
+> This vendor specific FileType allows the change of the RouterOS default configuration script that is executed when /system reset-configuration command is executed (or the other means when router configuration is beeing reset).
+
+## TR069 preset
+
+In TeamsACS, a Tr069 preset is a pre-configured TR069 RPC operation described in a yaml format file, which includes the RPC method to be executed, parameters, error handling methods, etc. Tr069 presets can be executed manually or triggered by TR069 specific events such as a boot event, or by a backend timed task in TeamsACS.
+
+### TR069 preset task
+
+When the TR069 preset execution is triggered, the TeamsACS system creates a task to track the execution of the TR069 preset. During execution, the system records the content and status of the execution, making it easy for managers to monitor and evaluate the execution process. This helps managers to have a better understanding of the status of the configuration being issued and to be able to address any issues in a timely manner.
+
+
+
+

docs/documents/RouterOS Radius Integration Guide.md

@@ -0,0 +1,72 @@
+## Mikrotik TR069 Client Setup for ToughRADIUS
+
+Here is the guide to configure ROUTEROS to integrate with TOUGHRADIUS:
+
+Firstly, we need to execute the following commands on ROUTEROS:
+
+```
+# Create address pool
+/ip pool
+add name=dhcp ranges=192.168.1.100-192.168.1.200
+
+# Create profile
+/ppp profile
+add name=radius local-address=192.168.1.1 remote-address=dhcp
+
+# Configure RADIUS server
+/radius
+add service=ppp,hotspot address=Radius_Server_IP secret=Radius_Secret authentication-port=1812 accounting-port=1813
+
+# Configure accounting interval
+/radius incoming
+set accept=yes port=3799
+
+# Configure PPP
+/ppp aaa
+set accounting=yes interim-update=2m use-radius=yes
+
+```
+
+Next, we need to create a corresponding VPE device on toughradius:
+
+```json
+{
+  "ID": 1,
+  "NodeId": 1,
+  "LdapId": 0,
+  "Name": "RouterOS",
+  "Identifier": "RouterOS",
+  "Hostname": "RouterOS Host",
+  "Ipaddr": "RouterOS IP",
+  "Secret": "Radius Secret",
+  "CoaPort": 3799,
+  "Model": "RouterOS",
+  "VendorCode": "14988",
+  "Status": "enabled",
+  "Tags": "",
+  "Remark": "",
+  "CreatedAt": "2022-06-01T08:00:00.000Z",
+  "UpdatedAt": "2022-06-01T08:00:00.000Z"
+}
+```
+
+Lastly, we carry out testing:
+
+> you also need to create the user account on the TOUGHRADIUS system
+
+Execute the following command on ROUTEROS to create a new PPP user:
+
+```
+/ppp secret
+add name=testuser password=testpass profile=radius service=pppoe
+
+```
+
+Configure a PPPoE connection on your client device, with the username testuser and the password testpass.
+
+Connect to the PPPoE, check the ROUTEROS and toughradius logs to confirm that the user's authentication and accounting information are being transmitted correctly.
+
+If the connection is successful, you should see the online status and usage of the user on the toughradius admin interface.
+
+You may also try to change the bandwidth limit for the user or disconnect the user on toughradius and check if the CoA (Change of Authorization) feature takes effect on ROUTEROS.
+

docs/documents/TR069 Preset template.md

@@ -0,0 +1,70 @@
+## ToughRADIUS 定义的 TR069 预置模板
+
+The description format of presets is the standard YAML format, which can facilitate the use of various data structures. For example, online automatic initialization of new devices can be done using a set of presets
+
+- oid  Can be set in scripts, factory settings, firmware configuration
+- enabled Indicates whether to enable this task
+- delay  For download tasks, the CPE can be delayed
+- onfail  If it is defined as cancel, when the task fails, all unexecuted tasks defined by the description file will be canceled; when defined as ignore, unexecuted tasks will continue to be executed
+
+The order of execution is  FactoryresetConfig -> FirmwareConfig -> Downloads ->Uploads -> SetParameterValues -> GetParameterNames,
+
+In a set of preset tasks, only some operations can be set, if the preset is executed by the system scheduled task,factoryreset, firmwareconfig will be ignored
+
+If the preset is performed by a scheduled system task (the time policy is set to `sys_scheduled`), then factoryreset, firmwareconfig are ignored in the set of preset tasks.
+
+
+```yaml
+# TR069 The default description format is the standard YAML format, which can facilitate the use of various data structures. For example, a set of presets can be used to complete the automatic initialization of new devices online
+
+#	- oid  Can be set in scripts, factory settings, firmware configuration
+#	- enabled Indicates whether to enable this task
+#	- delay  For download tasks, the CPE can be delayed
+#	- onfail  If it is defined as cancel, when the task fails, all unexecuted tasks defined by the description file will be canceled; when defined as ignore, unexecuted tasks will continue to be executed
+
+# The order of execution is  FactoryresetConfig -> FirmwareConfig -> Downloads ->Uploads -> SetParameterValues -> GetParameterNames,
+
+# If the preset is performed by a scheduled system task (the time policy is set to `sys_scheduled`), then factoryreset, firmwareconfig are ignored in the set of preset tasks.
+
+# Factory settings script description, single definition
+FactoryResetConfig:
+  oid: "test_factory_reset_cfg"
+  enabled: false
+  delay: 0
+  onfail: "ignore"
+
+# Firmware configuration description
+FirmwareConfig:
+  oid: "test_firmware_cfg"
+  enabled: false
+  delay: 0
+  onfail: "ignore"
+
+# Regular download script, supports multiple sequential execution
+Downloads:
+  - oid: "test_download"
+    enabled: true
+    delay: 0
+    onfail: "ignore"
+  - oid: "test_download2"
+    enabled: true
+    delay: 0
+    onfail: "ignore"
+
+# Regular upload tasks that support multiple sequential executions
+Uploads:
+  - filetype: "2 VendorLog File"
+    enabled: false
+    onfail: "ignore"
+
+# Set parameters, send multiple sets of parameters at one time
+SetParameterValues:
+  - name: "Device.DeviceInfo.X_MIKROTIK_SystemIdentity"
+    type: "string"
+    value: "TestRos"
+
+# Get parameters, support multiple sequential execution
+GetParameterValues:
+  - "Device.DeviceInfo."
+
+```

docs/documents/bras.md

@@ -0,0 +1,84 @@
+## Introduction to BRAS
+
+Broadband Remote Access Server (BRAS) is a high-capacity network device typically located in a service provider's (ISP's) network. It plays a crucial role in delivering high-speed internet access services, particularly in DSL or cable internet services.
+
+## Role of BRAS
+
+A BRAS is primarily used to manage user sessions and routing information, enabling users to access the internet. It serves as an intermediary between the user access network (like DSL or cable) and the service provider's core network. Upon connection of a user device (like a personal computer or a router) to the network, the BRAS carries out authentication, assigns IP addresses, and facilitates network access.
+
+In the ToughRADIUS system, a VPE device assumes the role of a BRAS, delivering these critical functionalities.
+
+### Functions of BRAS
+
+Here are some of the primary functions of a BRAS:
+
+User Authentication: The BRAS uses protocols like RADIUS or DIAMETER to verify the credentials of users and determine whether they should be allowed access to the network.
+
+IP Address Management: Once a user is authenticated, the BRAS assigns one or more IP addresses to the user, enabling them to communicate within the network.
+
+Session Management: The BRAS keeps track of each user's session, including the start and end times of the session, the amount of data transmitted, etc. This data can be used for billing, traffic management, and troubleshooting purposes.
+
+Routing: The BRAS also handles the routing of user traffic, ensuring that data packets are accurately delivered from their source to their destination.
+
+Quality of Service (QoS) Management: The BRAS can set and enforce QoS policies to ensure fair use of network resources and to meet the performance needs of different users and applications.
+
+From the above, it is clear that the VPE, acting as a BRAS device in the ToughRADIUS system, is a key device for users to access the network and engage in network communication.
+
+
+## VPE Model Definition
+
+The VPE model is a Go language struct, defined as follows:
+
+```golang
+type NetVpe struct {
+	ID         int64     `json:"id,string" form:"id"`            // Primary ID
+	NodeId     int64     `json:"node_id,string" form:"node_id"`  // Node ID
+	LdapId     int64     `json:"ldap_id,string" form:"ldap_id"`  // LDAP ID
+	Name       string    `json:"name" form:"name"`               // Device name
+	Identifier string    `json:"identifier" form:"identifier"`   // Device Identifier - RADIUS
+	Hostname   string    `json:"hostname" form:"hostname"`       // Device host address
+	Ipaddr     string    `json:"ipaddr" form:"ipaddr"`           // Device IP
+	Secret     string    `json:"secret" form:"secret"`           // Device RADIUS Secret
+	CoaPort    int       `json:"coa_port" form:"coa_port"`       // Device RADIUS COA port
+	Model      string    `json:"model" form:"model"`             // Device model
+	VendorCode string    `json:"vendor_code" form:"vendor_code"` // Device vendor code
+	Status     string    `json:"status" form:"status"`           // Device status
+	Tags       string    `json:"tags" form:"tags"`               // Tags
+	Remark     string    `json:"remark" form:"remark"`           // Remark
+	CreatedAt  time.Time `json:"created_at"`                     // Created at
+	UpdatedAt  time.Time `json:"updated_at"`                     // Updated at
+}
+
+```
+
+
+## Field explanations:
+
+* ID: Unique identifier of the VPE device
+* NodeId: Node ID
+* LdapId: LDAP ID
+* Name: Name of the VPE device
+* Identifier: RADIUS identifier of the device
+* Hostname: Host address of the device
+* Ipaddr: IP address of the device
+* Secret: RADIUS Secret of the device
+* CoaPort: RADIUS COA port of the device
+* Model: Model of the device
+* VendorCode: Vendor code of the device
+* Status: Status of the device
+* Tags: Tags of the device
+* Remark: Remarks about the device
+* CreatedAt: When the device entry was created
+* UpdatedAt: When the device entry was last updated
+
+
+## VPE Creation
+
+The creation of a VPE device primarily involves:
+
+* Submit a form containing device information
+* Validate necessary fields are not empty (e.g., name, VendorCode, and Identifier)
+* Create the new VPE device in the database
+
+<img width="1262" alt="image" src="https://github.com/talkincode/toughradius/assets/377938/e37804e4-e047-481e-a3b6-12d8f4d4c092">
+