JarCasting
JarCasting JarCasting

protobuf-extension

Base project for Protobuf extension for governance toolbox

Лицензия

Лицензия
Категории

Категории

Protobuf Данные Data Structures
Группа

Группа

com.github.frtu.governance
Идентификатор

Идентификатор

ext-protobuf-pom
Последняя версия

Последняя версия

1.1.0
Дата

Дата
Тип

Тип

pom
Описание

Описание

protobuf-extension
Base project for Protobuf extension for governance toolbox

Скачать ext-protobuf-pom

Имя Файла Размер
ext-protobuf-pom-1.1.0.pom 4 KB
Обзор

Как подключить последнюю версию

<!-- https://jarcasting.com/artifacts/com.github.frtu.governance/ext-protobuf-pom/ -->
<dependency>
    <groupId>com.github.frtu.governance</groupId>
    <artifactId>ext-protobuf-pom</artifactId>
    <version>1.1.0</version>
    <type>pom</type>
</dependency>
// https://jarcasting.com/artifacts/com.github.frtu.governance/ext-protobuf-pom/
implementation 'com.github.frtu.governance:ext-protobuf-pom:1.1.0'
// https://jarcasting.com/artifacts/com.github.frtu.governance/ext-protobuf-pom/
implementation ("com.github.frtu.governance:ext-protobuf-pom:1.1.0")
'com.github.frtu.governance:ext-protobuf-pom:pom:1.1.0'
<dependency org="com.github.frtu.governance" name="ext-protobuf-pom" rev="1.1.0">
  <artifact name="ext-protobuf-pom" type="pom" />
</dependency>
@Grapes(
@Grab(group='com.github.frtu.governance', module='ext-protobuf-pom', version='1.1.0')
)
libraryDependencies += "com.github.frtu.governance" % "ext-protobuf-pom" % "1.1.0"
[com.github.frtu.governance/ext-protobuf-pom "1.1.0"]

Зависимости

compile (2)

Идентификатор библиотеки Тип Версия
org.slf4j : slf4j-api jar
org.projectlombok : lombok Необязательный jar

test (2)

Идентификатор библиотеки Тип Версия
ch.qos.logback : logback-classic jar
junit : junit jar

Модули Проекта

  • base-proto-pom

governance-toolbox

A toolbox for automation & docker images around metadata governance :

Static metdata

  • Schema registry : to store schema & versions

Dynamic metadata

  • InfluxDB : TSDB to store PiT (Point In Time) based metadata.

Archetype for Avro

  • avro : generate a base project for avro data model, generate & compile
  • plt-kafka : standalone multi modules project to Publish & Consume Kafka
  • plt-spark : standalone multi modules project to run Spark

Libraries

Metadata :

  • library-dot : Dot graph generator library

Data :

  • library-serdes : Serialization Deserialization of Avro <-> JSON
  • library-kafka : Wrap Avro SerDes pack into Kafka Publisher & Consumer

Maven Plugins

  • schema-maven-plugin:pojo2json : Allow to generate an Avro schema based on POJO (Interface or Implementation). See sample project.

  • schema-maven-plugin:avro2dot (since 0.3.6) : Allow to generate an Dot Graph from an Avro schema. See sample project.

  • schema-registries-maven-plugin:register : Register all Avro schema from a folder into Hortonworks Schema-registry

SerDes platform

Characteristics Avro Protobuf (source)
Dynamic typing YES NO (needs to compile)
Backward compatibilities Complex Easy
Support null values Medium (Complex data struct)
Performance Very good
Compactness Excellent
Current toolbox support Good Good

Docker images

Confluent Stack (Kafka, REST API, ...)

Here are the different URLs

  • Kafka Bootstrap server : broker:9092

In order to make it works, you will need to map 'broker' to 127.0.0.1 in /etc/hosts. You can call the script at docker-schema-registry.bash

kafkahost

Confluent Schema-registry & Landoop UI

It serves the schema-registry-ui from port 8001. A live version can be found at Landoop Demo of schema-registry-ui

FYI find the API specs at : Confluent Schema Registry APIs

Attention this require at least Docker Compose 2.1 to allow Docker images start with order & validation.

Confluent Kafka REST & Landoop Topic UI

Hortonworks Schema-registry

Using alternative Docker image. For more details :

InfluxDB & Grafana

Using official Docker image. Look at this link for more details on the port :

Версии библиотеки

Версия
1.1.0
1.0.1