Skip to content

Monitoring of nodes using prometheus #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 67 commits into from
May 22, 2025
Merged

Monitoring of nodes using prometheus #13

merged 67 commits into from
May 22, 2025

Conversation

dviejokfs
Copy link
Contributor

@dviejokfs dviejokfs commented May 12, 2025

Add monitoring using prometheus to these types of nodes:

  • Besu
  • Hyperledger Fabric Peer
  • Hyperledger Fabric Orderer

Add audit logs to comply with ISO 27001/ SOC 2 standards

dviejokfs and others added 12 commits May 10, 2025 13:46
- Updated the `installCmd` struct to support multiple network configuration files, improving flexibility for organization-specific setups.
- Implemented detailed logging for installation parameters, including chaincode, channel, organizations, and network configurations, enhancing debugging capabilities.
- Introduced a new helper function `readPEMOrPath` to streamline the processing of user certificates and keys from either PEM strings or file paths.
- Enhanced the `LoadFromFile` function in the network configuration parser to automatically read and populate user certificates and keys, improving usability.
- Updated the `UserCert`, `UserKey`, and `TLSCACerts` structs to include optional file path fields for better configuration management.

These changes improve the overall functionality and maintainability of the installation process, ensuring better handling of network configurations and user credentials.

Signed-off-by: dviejokfs <dviejo@kfs.es>
- Updated HTTP handler methods to utilize new service methods for retrieving Fabric network configurations, improving code clarity and maintainability.
- Renamed service methods for consistency, such as `GetChannelConfig` to `GetFabricChannelConfig`, aligning with the new naming conventions.
- Introduced new service methods for managing Fabric network operations, including `ImportFabricNetworkWithOrg`, `GetFabricCurrentChannelConfig`, and `GetFabricChainInfo`, enhancing the service layer's capabilities.
- Enhanced error handling and logging across the updated methods to ensure robust interactions and better debugging.
- Updated Swagger documentation to reflect the changes in API endpoints and response structures, improving API usability.

These changes enhance the overall functionality and maintainability of the Fabric network service, ensuring better management of network configurations and operations.

Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: dviejokfs <dviejo@kfs.es>
- Added a new API endpoint `/api/v1/metrics/node/{id}/query` to execute custom Prometheus queries with optional time range.
- Introduced `CustomQueryRequest` struct for handling query parameters, including start, end, and step.
- Enhanced the `Query` and `QueryRange` methods in the service layer to support custom queries for specific nodes.
- Updated Swagger documentation to reflect the new endpoint and request/response structures.
- Improved error handling in the metrics client to provide detailed responses for unexpected status codes.
- Added new UI components for displaying metrics data, including `MetricsCard` and `MetricsGrid`.

These changes improve the flexibility and usability of the metrics API, allowing users to perform custom queries and visualize the results effectively.

Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Introduced a new audit logging service to track and manage audit events within the application.
- Created HTTP handlers for listing and retrieving audit logs, including pagination and filtering options.
- Added middleware for logging HTTP requests and responses, capturing relevant details for audit purposes.
- Defined database models and SQL queries for storing and retrieving audit logs.
- Updated Swagger documentation to include new endpoints for audit log management.
- Enhanced the frontend with new pages for displaying audit logs and details.

These changes improve the application's ability to monitor and audit actions, ensuring better compliance and traceability.

Signed-off-by: dviejokfs <dviejo@kfs.es>
- Added a new `sessionId` field to the audit log structure, allowing tracking of user sessions in audit events.
- Updated Swagger documentation and generated files to reflect the inclusion of `sessionId`.
- Modified middleware to capture and log session IDs during HTTP requests.
- Enhanced database models and SQL queries to accommodate the new `sessionId` field.
- Updated frontend components to display session IDs in audit logs.

These changes improve the audit logging capabilities, providing better traceability and context for logged events.

Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: dviejokfs <dviejo@kfs.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
dviejokfs added 3 commits May 15, 2025 12:27
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
dviejokfs added 3 commits May 15, 2025 13:10
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
dviejokfs added 2 commits May 15, 2025 13:32
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
dviejokfs added 4 commits May 15, 2025 14:37
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
dviejokfs and others added 29 commits May 18, 2025 00:48
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Introduced a new `testnet` command to manage testnets for different blockchain networks.
- Implemented subcommands for Besu and Fabric testnets.
- Added pagination support for listing organizations, including query parameters for limit and offset.
- Updated organization-related API responses to include pagination details.
- Refactored organization listing components to utilize the new paginated response structure.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Added `keymanagement.go` for creating and retrieving cryptographic keys via API.
- Introduced `CreateBesuNetwork` function in `networks.go` to create Besu networks with proper error handling and response parsing.
- Enhanced `BesuTestnetRunner` in `besu.go` to manage the creation of Besu testnets, including key generation and node setup.
- Updated `FabricTestnetRunner` to allow dynamic node mode configuration.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Updated `BesuTestnetRunner` in `besu.go` to include new configuration parameters such as GasLimit, Difficulty, MixHash, Coinbase, Timestamp, and Nonce for Besu nodes.
- Added dynamic metrics port allocation for Besu nodes to improve flexibility.
- Enhanced `FabricTestnetRunner` in `fabric.go` to enforce a minimum of three orderers for consenters, ensuring proper network configuration.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Updated the E2E test workflow to include new testnet creation steps for Besu and Fabric, ensuring proper setup before running API tests.
- Added versioning support for Besu nodes in the `create.go` command, allowing specification of the Besu version during node creation.
- Refactored node response handling to utilize a new HTTP types package for better separation of concerns.
- Introduced external IP retrieval functionality to improve network configuration for Besu testnets.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
…eer nodes

- Updated e2e-tests.yaml to include Docker mode for chainlaunch.
- Refactored orderer.go to build Docker environment variables and start the orderer in a Docker container.
- Implemented Docker container management in orderer service.
- Enhanced peer.go to support Docker container creation and management.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Updated the block retrieval command to capture and check the HTTP status code.
- Added error handling to log the response body if the status code is not 200.
- Ensured successful retrieval of a block from the channel is logged with the correct status code.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Removed excessive logging in install.go to streamline output.
- Implemented external IP detection in fabric.go for improved network configuration.
- Added validation for x-source parameters in plugin deployment to ensure correctness.
- Enhanced error handling in plugin.go for better feedback during deployment failures.
- Introduced new types and methods for managing x-source fields and options in plugin types.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Updated the plugin manager to accept additional services and improve initialization.
- Enhanced x-source parameter validation and processing for better error handling.
- Introduced new x-source types and handlers for fabric keys and peers.
- Improved organization response structure to include additional key IDs.
- Updated Swagger documentation to reflect changes in API response structures.

Signed-off-by: dviejokfs <dviejo@kfs.es>
- Updated error handling in the plugin deployment process to provide more detailed feedback.
- Enhanced the layout of the plugins page for better responsiveness and visual clarity.
- Added a description field to the plugin cards for improved information display.
- Refactored the DeploymentModal to support new data structures and improve form handling.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
- Added functionality to join all peers and orderers to the Fabric network, with error handling for any failures during the process.
- Improved the layout of the plugins page to ensure better responsiveness and visual clarity by adjusting the grid column spans.

Signed-off-by: David VIEJO <dviejo@kungfusoftware.es>
Signed-off-by: dviejokfs <dviejo@kfs.es>
- Added new types for Documentation, Example, and TroubleshootingItem to improve API documentation clarity.
- Updated the Metadata structure to include author, license, repository, and tags for better plugin information.
- Enhanced the PluginDetailPage to display documentation and metadata in a tabbed interface for improved user experience.
- Refactored YamlViewer component to integrate with the new structure and provide a dialog for YAML viewing.

Signed-off-by: dviejokfs <dviejo@kfs.es>
- Introduced a new API endpoint to retrieve the current status of the Prometheus instance, including version, port, and configuration details.
- Implemented functionality to fetch committed chaincodes for specific channels on Fabric peer nodes.
- Updated the Swagger documentation to reflect the new API endpoints and their responses.
- Enhanced the frontend to display analytics and committed chaincodes, improving user experience and accessibility of metrics.

Signed-off-by: dviejokfs <dviejo@kfs.es>
@dviejokfs dviejokfs merged commit 9a98269 into main May 22, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant