Skip to content

Monitoring of nodes using prometheus #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 67 commits into from
May 22, 2025
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
701abbd
Refactor install command and enhance network configuration handling
dviejokfs May 10, 2025
71c065f
Refactor Fabric network service methods and enhance API functionality
dviejokfs May 10, 2025
9d87c98
Add feedback URL
dviejokfs May 11, 2025
e1c6dc6
Fix events
dviejokfs May 11, 2025
14acb18
Initial backend implementation for metrics with prometheus
dviejokfs May 11, 2025
69532bb
Implement custom Prometheus query endpoint and enhance metrics handling
dviejokfs May 12, 2025
e45304f
Checkpoint
dviejokfs May 12, 2025
aedee9b
Update
dviejokfs May 12, 2025
7bfd294
Fix tests
dviejokfs May 12, 2025
a526408
Implement audit logging functionality
dviejokfs May 13, 2025
4eb4c9f
Enhance audit logging with session ID support
dviejokfs May 13, 2025
3e6279c
Start e2e tests
dviejokfs May 14, 2025
f490433
Update
dviejokfs May 15, 2025
da41b90
Update tests
dviejokfs May 15, 2025
8fc33d3
Update
dviejokfs May 15, 2025
ba1fc38
Update
dviejokfs May 15, 2025
5d83258
Update
dviejokfs May 15, 2025
c11338e
Update
dviejokfs May 15, 2025
58cd2f6
Update
dviejokfs May 15, 2025
7584897
Update
dviejokfs May 15, 2025
a842328
Update
dviejokfs May 15, 2025
ce4a9f4
Update
dviejokfs May 15, 2025
6999f3c
Update
dviejokfs May 15, 2025
d65f8e2
Fix tests
dviejokfs May 15, 2025
82a71fa
Update
dviejokfs May 15, 2025
bfaeb88
Refactor API availability checks in workflows and update Playwright c…
dviejokfs May 15, 2025
4ce4d1e
Fix tests
dviejokfs May 16, 2025
38772a7
Update tests
dviejokfs May 16, 2025
adf4af0
Update
dviejokfs May 16, 2025
aac03bb
Update
dviejokfs May 16, 2025
f92351c
Update
dviejokfs May 16, 2025
c7020b5
Merge e2e tests
dviejokfs May 16, 2025
349bc81
Update
dviejokfs May 16, 2025
837254b
Update
dviejokfs May 16, 2025
9d492de
Update
dviejokfs May 16, 2025
8be1594
Update
dviejokfs May 16, 2025
bbcf835
Update
dviejokfs May 16, 2025
4d2e6d2
Add Playwright tests for creating Fabric nodes and user management
dviejokfs May 17, 2025
afb4ca1
Update
dviejokfs May 17, 2025
5b5d780
Update
dviejokfs May 18, 2025
704d4a8
Update
dviejokfs May 18, 2025
50a021c
Add testnet command and organization pagination support
dviejokfs May 18, 2025
8d0fdfd
Implement key management and Besu network creation in Go
dviejokfs May 18, 2025
2d372fb
Enhance Besu and Fabric testnet runners with additional configurations
dviejokfs May 18, 2025
d32dd78
Enhance E2E testing and Besu node configuration
dviejokfs May 19, 2025
330452e
Update
dviejokfs May 19, 2025
8e349a8
ci/cd
dviejokfs May 19, 2025
792d945
Cache build
dviejokfs May 19, 2025
2ebf300
Update
dviejokfs May 19, 2025
54c1039
Update
dviejokfs May 19, 2025
936f377
ci/cd
dviejokfs May 19, 2025
8ff7e24
ci/cd
dviejokfs May 19, 2025
7c90de1
ci/cd
dviejokfs May 19, 2025
012162f
ci/cd
dviejokfs May 19, 2025
51d13f7
ci/cd
dviejokfs May 19, 2025
d36ee61
ci/cd
dviejokfs May 19, 2025
b62daf0
Enhance CI/CD workflow and implement Docker support for orderer and p…
dviejokfs May 19, 2025
11c2374
ci/cd
dviejokfs May 19, 2025
94ac807
Enhance e2e-tests.yaml to improve error handling in block retrieval
dviejokfs May 19, 2025
b60b47f
Refactor logging and enhance error handling in various components
dviejokfs May 21, 2025
ebfea7e
Refactor plugin management and enhance x-source handling
dviejokfs May 21, 2025
9c9e0e6
Improve error handling and UI enhancements in plugin pages
dviejokfs May 21, 2025
4936956
Enhance Fabric network joining process and improve plugins page layout
dviejokfs May 21, 2025
dc4d426
Update
dviejokfs May 22, 2025
a4a1b0d
Merge branch 'feat/keep-going' of https://github.com/LF-Decentralized…
dviejokfs May 22, 2025
bd9e554
Update documentation structure and enhance plugin metadata
dviejokfs May 22, 2025
842669d
Add Prometheus status endpoint and chaincode retrieval for Fabric peers
dviejokfs May 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions .github/workflows/e2e-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: End-to-End Tests

on:
push:
branches:
- main
pull_request:
branches:
- main
paths-ignore:
- '**/*.md'
- 'docs/**'

jobs:
e2e-tests:
name: Run E2E Tests
runs-on: ubuntu-latest

services:
# Mailhog for email testing
mailhog:
image: mailhog/mailhog
ports:
- 1025:1025 # SMTP server
- 8025:8025 # Web interface

env:
API_BASE_URL: http://localhost:8100/api/v1
CYPRESS_BASE_URL: http://localhost:8100
CYPRESS_API_URL: http://localhost:8100/api/v1
CHAINLAUNCH_USER: admin
CHAINLAUNCH_PASSWORD: admin123

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23.4'
cache: true

- name: Setup Bun
uses: oven-sh/setup-bun@v1

- name: Install dependencies
run: |
go mod download
sudo apt-get update
sudo apt-get install -y build-essential

- name: Build the application
run: |
go build -v -o chainlaunch ./main.go
chmod +x chainlaunch

- name: Start the application
run: |
# Create test data directory
mkdir -p testdata

# Start the application in the background
./chainlaunch serve --db data.db &
sleep 10 # Wait for the application to start

- name: Run API tests
run: |
go test -v ./tests/e2e/... -tags=e2e

- name: Run UI tests
run: |
cd web
bun install
bun run test:e2e

- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results
path: |
web/cypress/videos
web/cypress/screenshots
test-results.xml
2 changes: 1 addition & 1 deletion .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
"env": {
"CHAINLAUNCH_USER": "admin",
"CHAINLAUNCH_PASSWORD": "admin123",
"JAVA_HOME": "/opt/homebrew/opt/openjdk@21"
"JAVA_HOME": "/opt/homebrew/Cellar/openjdk/23.0.2"
}
},
{
Expand Down
4 changes: 2 additions & 2 deletions cmd/common/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ func NewClientFromEnv() (*Client, error) {
apiURL = defaultAPIURL
}

username := os.Getenv("CHAINLAUNCH_USERNAME")
username := os.Getenv("CHAINLAUNCH_USER")
if username == "" {
return nil, fmt.Errorf("CHAINLAUNCH_USERNAME environment variable is not set")
return nil, fmt.Errorf("CHAINLAUNCH_USER environment variable is not set")
}

password := os.Getenv("CHAINLAUNCH_PASSWORD")
Expand Down
158 changes: 112 additions & 46 deletions cmd/fabric/install/install.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ import (
"path/filepath"

"github.com/chainlaunch/chainlaunch/pkg/logger"
"google.golang.org/protobuf/proto"
"github.com/pkg/errors"
"google.golang.org/grpc"
"google.golang.org/protobuf/proto"

"io"
"strings"
Expand All @@ -32,7 +32,7 @@ import (
type installCmd struct {
chaincode string
channel string
networkConfig string
networkConfigs []string
users []string
organizations []string
signaturePolicy string
Expand Down Expand Up @@ -64,14 +64,21 @@ func (c *installCmd) getPeerAndIdentityForOrg(nc *networkconfig.NetworkConfig, o
if !ok {
return nil, nil, fmt.Errorf("user %s not found in network config", userID)
}

// Get user certificate
userCert, err := gwidentity.CertificateFromPEM([]byte(user.Cert.PEM))
if err != nil {
return nil, nil, errors.Wrapf(err, "failed to read user certificate for user %s and org %s", userID, org)
}

// Get user private key
userPrivateKey, err := gwidentity.PrivateKeyFromPEM([]byte(user.Key.PEM))
if err != nil {
return nil, nil, errors.Wrapf(err, "failed to read user private key for user %s and org %s", userID, org)
}
// Log certificate and private key PEM content for debugging
c.logger.Infof("User certificate for %s in org %s: %s", userID, org, user.Cert.PEM)
c.logger.Infof("User private key for %s in org %s: %s", userID, org, user.Key.PEM)
userIdentity, err := identity.NewPrivateKeySigningIdentity(org, userCert, userPrivateKey)
if err != nil {
return nil, nil, errors.Wrapf(err, "failed to create user identity for user %s and org %s", userID, org)
Expand Down Expand Up @@ -120,27 +127,103 @@ func (c installCmd) start() error {
_ = pkg
packageID := chaincode.GetPackageID(label, pkg)
c.logger.Infof("packageID: %s", packageID)
nc, err := networkconfig.LoadFromFile(c.networkConfig)
if err != nil {
return err

// Load network configs
networkConfigs := make([]*networkconfig.NetworkConfig, len(c.organizations))
if len(c.networkConfigs) == 0 {
return fmt.Errorf("at least one network config file is required")
}

// // install chaincode in peers
// configBackend := config.FromFile(c.networkConfig)

// clientsMap := map[string]*resmgmt.Client{}
// sdk, err := fabsdk.New(configBackend)
// if err != nil {
// return err
// }
// for idx, mspID := range c.organizations {
// clientContext := sdk.Context(fabsdk.WithUser(c.users[idx]), fabsdk.WithOrg(mspID))
// clientsMap[mspID], err = resmgmt.New(clientContext)
// if err != nil {
// return err
// }
// }
// If only one config is provided, use it for all organizations
if len(c.networkConfigs) == 1 {
nc, err := networkconfig.LoadFromFile(c.networkConfigs[0])
if err != nil {
return err
}
for i := range c.organizations {
networkConfigs[i] = nc
}
} else {
// If multiple configs are provided, validate the count matches organizations
if len(c.networkConfigs) != len(c.organizations) {
return fmt.Errorf("number of network configs (%d) must match number of organizations (%d)", len(c.networkConfigs), len(c.organizations))
}
// Load each config
for i, configPath := range c.networkConfigs {
nc, err := networkconfig.LoadFromFile(configPath)
if err != nil {
return fmt.Errorf("failed to load network config %s: %w", configPath, err)
}
networkConfigs[i] = nc
}
}
// Log debug information about the installation parameters
c.logger.Infof("Debug information for chaincode installation:")
c.logger.Infof("Chaincode: %s", c.chaincode)
c.logger.Infof("Channel: %s", c.channel)
c.logger.Infof("Package ID: %s", packageID)

// Log organizations
c.logger.Infof("Organizations (%d):", len(c.organizations))
for i, org := range c.organizations {
c.logger.Infof(" [%d] %s", i, org)
}

// Log network configs
c.logger.Infof("Network configs (%d):", len(c.networkConfigs))
for i, configPath := range c.networkConfigs {
c.logger.Infof(" [%d] %s", i, configPath)
}

// Log users
c.logger.Infof("Users (%d):", len(c.users))
for i, user := range c.users {
c.logger.Infof(" [%d] %s", i, user)
}

// Log detailed network configuration information
for i, nc := range networkConfigs {
orgName := ""
if i < len(c.organizations) {
orgName = c.organizations[i]
}
c.logger.Infof("Network config [%d] details for org %s:", i, orgName)

if orgName != "" {
if orgConfig, ok := nc.Organizations[orgName]; ok {
c.logger.Infof(" MSPID: %s", orgConfig.MSPID)
c.logger.Infof(" Peers (%d):", len(orgConfig.Peers))
for j, peerID := range orgConfig.Peers {
if peerConfig, ok := nc.Peers[peerID]; ok {
c.logger.Infof(" [%d] %s - URL: %s", j, peerID, peerConfig.URL)
} else {
c.logger.Infof(" [%d] %s - Not found in peers configuration", j, peerID)
}
}
c.logger.Infof(" Users (%d):", len(orgConfig.Users))
for userName := range orgConfig.Users {
c.logger.Infof(" %s", userName)
}
} else {
c.logger.Infof(" Organization %s not found in network config", orgName)
}
}
}

// Log other parameters
c.logger.Infof("Chaincode address: %s", c.chaincodeAddress)
c.logger.Infof("Signature policy: %s", c.signaturePolicy)
c.logger.Infof("Local mode: %v", c.local)
if c.pdcFile != "" {
c.logger.Infof("PDC file: %s", c.pdcFile)
}
if c.metaInfPath != "" {
c.logger.Infof("META-INF path: %s", c.metaInfPath)
}

// Install chaincode in peers
for idx, org := range c.organizations {
nc := networkConfigs[idx]
orgConfig, ok := nc.Organizations[org]
if !ok {
return fmt.Errorf("organization %s not found in network config", org)
Expand Down Expand Up @@ -168,10 +251,6 @@ func (c installCmd) start() error {
}
}

// sp, err := policydsl.FromString(c.signaturePolicy)
// if err != nil {
// return err
// }
applicationPolicy, err := chaincode.NewApplicationPolicy(c.signaturePolicy, "")
if err != nil {
return err
Expand All @@ -181,6 +260,7 @@ func (c installCmd) start() error {
sequence := 1
allOrgGateways := []*chaincode.Gateway{}
for idx, org := range c.organizations {
nc := networkConfigs[idx]
orgConfig, ok := nc.Organizations[org]
if !ok {
return fmt.Errorf("organization %s not found in network config", org)
Expand Down Expand Up @@ -208,7 +288,6 @@ func (c installCmd) start() error {

var collections []*pb.CollectionConfig
if c.pdcFile != "" {
//
pdcBytes, err := ioutil.ReadFile(c.pdcFile)
if err != nil {
return err
Expand Down Expand Up @@ -277,18 +356,6 @@ func (c installCmd) start() error {
c.logger.Infof("Chaincode already committed, version=%s sequence=%d", version, sequence)
}
c.logger.Infof("Should commit=%v", shouldCommit)
// // approve chaincode in orgs
// approveCCRequest := resmgmt.LifecycleApproveCCRequest{
// Name: label,
// Version: version,
// PackageID: packageID,
// Sequence: int64(sequence),
// CollectionConfig: collections,
// EndorsementPlugin: "escc",
// ValidationPlugin: "vscc",
// SignaturePolicy: sp,
// InitRequired: false,
// }

chaincodeDef := &chaincode.Definition{
ChannelName: c.channel,
Expand All @@ -305,17 +372,17 @@ func (c installCmd) start() error {
for idx, gateway := range allOrgGateways {
err := gateway.Approve(ctx, chaincodeDef)
if err != nil {
c.logger.Errorf("Error when approving chaincode: %v", err)
return err
}
if err != nil && !strings.Contains(err.Error(), "redefine uncommitted") {
c.logger.Errorf("Error when approving chaincode: %v", err)
return err
if strings.Contains(err.Error(), "redefine uncommitted") {
c.logger.Infof("Chaincode already committed, org=%s", c.organizations[idx])
} else {
c.logger.Errorf("Error when approving chaincode: %v", err)
return err
}
}

c.logger.Infof("Chaincode approved, org=%s", c.organizations[idx])
}
if shouldCommit {

// commit chaincode in orgs
err := firstGateway.Commit(
ctx,
Expand All @@ -326,7 +393,6 @@ func (c installCmd) start() error {
return err
}
c.logger.Infof("Chaincode committed")

}

if c.envFile != "" {
Expand Down Expand Up @@ -546,7 +612,7 @@ func NewInstallCmd(logger *logger.Logger) *cobra.Command {
f := cmd.Flags()
f.StringVar(&c.chaincode, "chaincode", "", "chaincode name within the channel")
f.StringVar(&c.channel, "channel", "", "Channel name")
f.StringVar(&c.networkConfig, "config", "", "Network config file")
f.StringArrayVar(&c.networkConfigs, "config", []string{}, "Network config files (one per organization, if only one is provided it will be used for all organizations)")
f.StringVar(&c.signaturePolicy, "policy", "", "Signature policy for the chaincode")
f.StringArrayVarP(&c.organizations, "organizations", "o", []string{}, "Organizations to connect to ")
f.StringArrayVarP(&c.users, "users", "u", []string{}, "Users to use")
Expand Down
Loading
Loading