Skip to content

replace Map with sync.Map to handle concurrent reads/writes (changes generated by Cursor); #2646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

jaredholloway94
Copy link

Replace Map with sync.Map to handle concurrent reads/writes

Description

replace Map with sync.Map to handle concurrent reads/writes


Checklist

Before submitting your pull request, ensure the following requirements are met:

  • Code follows the PEP 8 style guide.
  • Code has been formatted with Black using the command:
    pipenv run black {source_file_or_directory}
  • Changes are tested and verified to work as expected.

Related Issues

If applicable, link the issues resolved by this pull request:

  • Resolves #[2610]

Additional Notes

These code changes were generated by Cursor AI.
I don't know Go that well, but these changes seem to work in local testing.


Thank you for contributing to pyRevit! 🎉

Copy link
Contributor

@devloai devloai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary:

This PR replaces standard Go maps with sync.Map in the telemetry server to handle concurrent reads/writes. The changes affect the data models and initialization code in the server handlers. The goal is to prevent potential race conditions that could occur with regular maps in a concurrent environment.

Review Summary:

While using sync.Map is a good approach for concurrent access, there are significant implementation issues that need to be addressed. The most critical problem is that Go's standard JSON encoder doesn't properly support sync.Map, which could lead to data loss or corruption. Additionally, there are type safety concerns and incomplete map conversion code in some methods.

Please let me know if you have any questions about implementing these fixes - I'd be happy to provide more guidance!

Follow-up suggestions:

@devloai implement custom JSON marshaling/unmarshaling for the structs using sync.Map

@@ -37,7 +38,9 @@ func RouteEvents(router *mux.Router, opts *cli.Options, dbConn persistence.Conne
// https://stackoverflow.com/a/26212073
router.HandleFunc("/api/v2/events/", func(w http.ResponseWriter, r *http.Request) {
// parse given json data into a new record
logrec := persistence.EventTelemetryRecordV2{}
logrec := persistence.EventTelemetryRecordV2{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sync.Map initialization is correct, but there should be proper error handling for JSON deserialization of sync.Map fields. If the incoming JSON contains data intended for EventArgs, the current code might silently ignore it since the unmarshalled data may not properly map to sync.Map. Consider adding custom JSON unmarshalling for this struct or validate that EventArgs is correctly populated after decoding.

}

func (logrec ScriptTelemetryRecordV2) PrintRecordInfo(logger *cli.Logger, message string) {
// Convert sync.Map to regular map for printing
results := make(map[string]interface{})
logrec.CommandResults.Range(func(key, value interface{}) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a potential type safety issue here. The original code has explicit type assertions when converting to a regular map, but the current implementation might assign different types to the results map. The value type should be explicitly checked like it is in the ScriptTelemetryRecordV1.PrintRecordInfo method (lines 46-54). Consider adding proper type assertion for the value parameter similar to:

logrec.CommandResults.Range(func(key, value interface{}) bool {
    if k, ok := key.(string); ok {
        if v, ok := value.(interface{}); ok {
            results[k] = v
        }
    }
    return true
})

BundleName string `json:"commandbundle" bson:"commandbundle" valid:"-"`
ExtensionName string `json:"commandextension" bson:"commandextension" valid:"-"`
ResultCode int `json:"resultcode" bson:"resultcode" valid:"numeric~Invalid result code"`
CommandResults sync.Map `json:"commandresults" bson:"commandresults" valid:"-"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using sync.Map with JSON marshaling/unmarshaling may cause issues with serialization. The default Go JSON encoder doesn't have built-in support for sync.Map. You'll need to implement custom MarshalJSON and UnmarshalJSON methods for each struct using sync.Map to properly serialize and deserialize the data. Without this, data might be lost or improperly stored during JSON operations.

TimeStamp string `json:"timestamp" bson:"timestamp" valid:"rfc3339~Invalid timestamp"`
HandlerId string `json:"handler_id" bson:"handler_id" valid:"-"`
EventType string `json:"type" bson:"type" valid:"-"`
EventArgs sync.Map `json:"args" bson:"args" valid:"-"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The EventTelemetryRecordV2 PrintRecordInfo method (line 204) doesn't handle the EventArgs sync.Map conversion before printing. Add code similar to what's done in ScriptTelemetryRecordV2.PrintRecordInfo to convert the sync.Map to a regular map before printing, otherwise the logging output may be incorrect or misleading.

@jmcouffin
Copy link
Contributor

Thanks Jared,
I am not literate in GO (like, at all!)
Adn I haven't used the telemetry in years, could you provide steps for me to test it locally? @jaredholloway94

@jaredholloway94
Copy link
Author

jaredholloway94 commented Apr 16, 2025

@jmcouffin

Setup with Docker:

  1. install Docker
  2. git clone https://github.com/jaredholloway94/docker-pyRevitTelemetry.git
  3. in docker-pyRevitTelemetry/telemetry_server/Dockerfile on line 9, replace https://github.com/pyrevitlabs/pyRevit.git with https://github.com/jaredholloway94/pyRevit.git
  4. follow the rest of the Installation instructions here: https://github.com/jaredholloway94/docker-pyRevitTelemetry

Setup without Docker:

  1. install Go
  2. install Postgres
  3. follow instructions to create database and tables with correct schema: scripts events
  4. git clone https://github.com/pyrevitlabs/pyRevit.git
  5. cd pyRevit/dev/pyRevitTelemetryServer
  6. go run main.go postgres://[YOUR_POSTGRES_USER]:[YOUR_POSTGRES_PASSWORD]@[YOUR_POSTGRES_HOST]:5432/[YOUR_POSTGRES_DB_NAME]?sslmode=disable --scripts=[YOUR_SCRIPTS_TABLE_NAME] --events=[YOUR_EVENTS_TABLE_NAME] --port=8080

After setup:

  1. start Revit
  2. in pyRevit settings, enable script and event telemetry
  3. point telemetry at your test server as described here: https://pyrevitlabs.notion.site/Telemetry-System-992d72659457447f86b79cf1c9034541#c44dd22f74bc45e591c9834131421906
  4. Open a model in Revit, and watch console output to see events hit the telemetry server (if using Docker, docker logs <container id> to see the output).

If you want to see the event records hit the database:

  1. install pgAdmin
  2. right-click Servers > Register Server... > Connection tab > enter hostname/address and password
  3. once connected, under Databases > [YOUR_DB_NAME] > Schemas > Tables > [YOUR_EVENTS_TABLE_NAME] > right-click > View/Edit Data > All Rows

let me know if you get stuck.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants