JSON Schema references

[dunivlorraine]

Issue submitter TODO list

• I've searched for an already existing issues here
• I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?

Hi,

I have a Kafka-Registry instance that I can manage with Kafbat. I have multiple schemas, and I want to create references from one another. As stated in the Confluent Kafka Registry documentation, a JSON Schema reference ($ref) should be a URL pointing to the referenced schema base URI.

Because of #476, we cannot use IDs to reference other schemas. This leads us with the retrieval URI, which is not documented anywhere. Also, it seems that when we enter a JSON Schema, only "schema" part can be modified from the UI, not the "reference" part. I could send HTTP requests to the Kafka Registry API and add these references, but that's just not practical at all.

So how I am supposed to reference another schema in the same (or another) registry from the UI? There is another similar issue but it lacks context: #49. Maybe I've misundersand how Kafka Schema Registry works, I've already used JSON Schema quite a lot, but I can't figure how to do this basic thing.

Thanks!

Describe the feature you're interested in

Add a way to manage (list, add, remove) JSON Schema references from the UI.

Describe alternatives you've considered

No response

Version you're running

latest (4cf17a0 v1.1.0)

Additional context

Here's my Docker compose file:

services:
  kafka0:
    container_name: kafka0
    image: confluentinc/cp-kafka:7.5.3
    ports:
      - "9092:9092"
    environment:
      # Required Kraft properties
      CLUSTER_ID: "b8OKBMz4S5C6TAAAls0yGw" # This MUST be unique accross all topics and clusters IDs.
      KAFKA_PROCESS_ROLES: 'broker,controller'
      KAFKA_NODE_ID: 1
      KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka0:9093'
      # Listeners
      KAFKA_INTER_BROKER_LISTENER_NAME: BROKER
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENERS: PLAINTEXT_HOST://0.0.0.0:9092,BROKER://:29092,CONTROLLER://:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT_HOST:PLAINTEXT,BROKER:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT_HOST://localhost:9092,BROKER://kafka0:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

  kafka-ui:
    depends_on:
      - kafka0
      - schemaregistry0
      - kafka-connect0
    container_name: kafka-ui
    image: ghcr.io/kafbat/kafka-ui:latest # v1.1.0
    ports:
      - 8080:8080
    environment:
      KAFKA_CLUSTERS_0_NAME: local
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka0:29092
      DYNAMIC_CONFIG_ENABLED: 'true'
      # Schema registry
      KAFKA_CLUSTERS_0_SCHEMAREGISTRY: http://schemaregistry0:8085
      # Kafka Connect
      KAFKA_CLUSTERS_0_KAFKACONNECT_0_NAME: first
      KAFKA_CLUSTERS_0_KAFKACONNECT_0_ADDRESS: http://kafka-connect0:8083
    
  schemaregistry0:
    image: confluentinc/cp-schema-registry:7.2.1
    ports:
      - 8085:8085
    depends_on:
      - kafka0
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: PLAINTEXT://kafka0:29092
      SCHEMA_REGISTRY_KAFKASTORE_SECURITY_PROTOCOL: PLAINTEXT
      SCHEMA_REGISTRY_HOST_NAME: schemaregistry0
      SCHEMA_REGISTRY_LISTENERS: http://schemaregistry0:8085
      SCHEMA_REGISTRY_SCHEMA_REGISTRY_INTER_INSTANCE_PROTOCOL: "http"
      SCHEMA_REGISTRY_LOG4J_ROOT_LOGLEVEL: INFO
      SCHEMA_REGISTRY_KAFKASTORE_TOPIC: _schemas
  
  kafka-connect0:
    image: confluentinc/cp-kafka-connect:7.2.1
    ports:
      - 8083:8083
    depends_on:
      - kafka0
      - schemaregistry0
    environment:
      CONNECT_BOOTSTRAP_SERVERS: kafka0:29092
      CONNECT_GROUP_ID: compose-connect-group
      CONNECT_CONFIG_STORAGE_TOPIC: _connect_configs
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_STORAGE_TOPIC: _connect_offset
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_TOPIC: _connect_status
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_REST_ADVERTISED_HOST_NAME: kafka-connect0
      # Converters
      CONNECT_INTERNAL_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_INTERNAL_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      # Schema registry
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: http://schemaregistry0:8085
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.storage.StringConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schemaregistry0:8085
      # Plugins
      CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components"

[github-actions]

Hi dunivlorraine! 👋

Welcome, and thank you for opening your first issue in the repo!

Please wait for triaging by our maintainers.

As development is carried out in our spare time, you can support us by sponsoring our activities or even funding the development of specific issues.
Sponsorship link

If you plan to raise a PR for this issue, please take a look at our contributing guide.

[germanosin]

Hi, @dunivlorraine,

Thanks for submitting a feature request! This is a fairly complex feature and may take significant time to implement. If anyone is interested in taking it on, I'd be happy to review it.