Skip to content

Commit ae4bf1a

Browse files
paulparkinsonRichardExleyrenagranat
authored
add mongodb postgred kafka services and setup #57 and fix for inventory-dotnet DB passwords #76 (#123)
* fixes * Update core.tf * fixes * Update oke-setup.sh * fixes * Boost Node CPU and Memory * Rename states for clarity * Update containerengine.tf * Update main-setup.sh * Build perf and main-setup.sh restart * fix * compute shape and docker login enhancements * Update main-setup.sh * build perf tuning * build fixes * Update build-all.sh * Update oci-cli-cs-key-auth.sh * terraform 3 AD * Update oci-cli-cs-key-auth.sh * Update core.tf * fixes * Update core.tf * fixes Uncomment Internet Gateway Comment DHCP settings Move SSL creation to earlier Add wait for OKE to warm up * fixes PropSetup Back to single domain Wait for OKE nodes msdataworkshop alias cwallet.sso name change remove terraform-orig * fixes wallet.zip VM.Standard.B1.1 image Prune terraform status to 60 characters * fixes add cwallet.sso link rm wallet folder (for security) * fixes Destroy new object atp secrets wait for OKE improved logging and setup handling * minor fixes * Fixes Remove RUN_TYPE for now Execute JAVA_BUILD and NON_JAVA_BUILD from main-setup * Fixes No Key Generated messages Improve end of setup * Update containerengine.tf Change image shape * Fixes Hardening oke-setup Different Shape * Update oke-setup.sh Hardening * Parallel Destroy Make destroy run in parallel * chmod * Update main-setup.sh More hardening * Update main-setup.sh * Update oke-setup.sh More defensive code * Typo * SODA build + cleanup Replace SODA jar with maven dependency Remove old scripts Fix global deploy/undeploy scripts * Reorg Jaeger * Jaeger Fixes * DB Prep in Setup and DB Name Changes Moved DB Setup from Java to Shell Change DB Name to o and i * Fixes and Start of GB Integration * DB Name Changes * Update db-setup.sh Fixes * dotnet inventorylocation query * Update main-setup.sh * dotnet inventorylocation query * Update db-setup.sh * GB Integration and Prop Fixes * fixes * Update oci-cli-cs-key-auth.sh * Removed DB Setup from UI * Update oci-cli-cs-key-auth.sh * Dot Net and Go * Fixes * Update main-setup.sh * Manage Non-Java Builds Better * inventory-go work * Update main-destroy.sh * inventory-go work * inventory-go aq dequeue added * Build Reorg * Update main-setup.sh Reorg password collection * Update main-setup.sh Minor fixes * Minor Fixes * Binding Changes * Fixes * Remove Legacy Code Removed admin-helidon, atp-setup, Removed initContainers from deployment yamls * inventory-go working with pl/sql * Fixed inventory-helidon-se * Minor fixes * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * Update db-setup.sh * Update main-setup.sh * Update main-setup.sh * Update db-setup.sh * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * Update db-setup.sh * Fixes go build.sh fix * Update java-builds.sh * Update non-java-builds.sh * Update oke-setup.sh * Update oke-setup.sh * Update db-setup.sh * Update non-java-builds.sh * Update java-builds.sh * Update non-java-builds.sh * Update java-builds.sh * Update db-setup.sh * Update oke-setup.sh * Update java-builds.sh * Update non-java-builds.sh * Update java-builds.sh * Update non-java-builds.sh * Update oke-setup.sh * Simpler Approach * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh Remove API Key based authentication as bug is fixed * Update main-destroy.sh Complete API Key removal * Update destroy.sh * Fix Inventory DotNet and Go * Update logback.xml Fixed inventory-helidon-se * Update db-setup.sh Protect password in it contains case special characters * dotnet and go services: switch to polling for msgs * Update main-setup.sh * Update main-setup.sh * Update main-setup.sh * verrazzano/multi-cloud work * verrazzano comp files * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * polyglot inventory service mods * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * verrazzano/multi-cloud work * inventory-springboot * inventory-helidon fix * mongodb install * mongodm * mongodb install * mongodm * mongodm * mongo-kafka-postgres work * mongodb-kafka-postgres work * mongo * postgres work * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongodm * mongo etc work * mongo kafka postgres work * mongo kafka postgres work * mongo kafka postgres work * removed sqlnet.ora edit * removed sqlnet.ora edit * removed sqlnet.ora edit * dotnet docker work and password/url fix * dotnet docker work and password/url fix * dotnet docker work and password/url fix * dotnet docker work and password/url fix * dotnet docker work and password/url fix * dotnet docker work and password/url fix * dotnet docker work and password/url fix * dotnet docker work and password/url fix * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * remove sqlnet.ora edit work * revert * revert * dotnet pw fix * dotnet pw fix * dotnet pw fix Co-authored-by: RichardExley <Richard.exley@oracle.com> Co-authored-by: irina granat <irina.granat@oracle.com>
1 parent eae5995 commit ae4bf1a

File tree

16 files changed

+97
-63
lines changed

16 files changed

+97
-63
lines changed

grabdish/inventory-dotnet/Startup.cs

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -81,22 +81,21 @@ public String ListenForMessages()
8181
// DbProviderFactory factory = DbProviderFactories.GetFactory("Oracle.ManagedDataAccess.Client"); DbCommand oracleCommand = factory.CreateCommand();
8282
String tnsAdmin = Environment.GetEnvironmentVariable("TNS_ADMIN");
8383
OracleConfiguration.WalletLocation = tnsAdmin;
84+
String pw = Environment.GetEnvironmentVariable("DB_PASSWORD");
8485
string connString =
8586
"User Id=" +
8687
Environment.GetEnvironmentVariable("DB_USER") +
8788
";Password=" +
88-
Environment.GetEnvironmentVariable("DB_PASSWORD") +
89+
"\"" + pw + "\"" +
8990
";Data Source=" +
9091
Environment.GetEnvironmentVariable("DB_CONNECT_STRING") +
9192
";";
92-
Console
93-
.WriteLine("tnsAdmin:" +
94-
tnsAdmin);
9593
using (
9694
OracleConnection connection = new OracleConnection(connString)
9795
)
9896
{
9997
connection.Open();
98+
Console.WriteLine("listening for messages...");
10099
while (true) {
101100
try
102101
{

grabdish/inventory-postgres-kafka/deploy.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,6 @@ if [ -z "$1" ]; then
2626
else
2727
kubectl apply -f <(istioctl kube-inject -f $SCRIPT_DIR/inventory-postgres-kafka-deployment-$CURRENTTIME.yaml) -n msdataworkshop
2828
fi
29+
kubectl delete service supplier -n msdataworkshop
30+
kubectl apply -f $SCRIPT_DIR/supplier-service.yaml -n msdataworkshop # temporarily for inventory adjustment, will use inventory service
31+
kubectl apply -f $SCRIPT_DIR/inventory-service.yaml -n msdataworkshop # for various including setting of failures and eventually inventory adjustment calls, etc.

grabdish/inventory-postgres-kafka/inventory-postgres-kafka-deployment.yaml

Lines changed: 11 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -20,22 +20,17 @@ spec:
2020
image: %DOCKER_REGISTRY%/inventory-postgres-kafka:0.1
2121
imagePullPolicy: Always
2222
env:
23-
- name: oracle.ucp.jdbc.PoolDataSource.inventorypdb.user
24-
value: "inventoryuser"
25-
- name: oracle.ucp.jdbc.PoolDataSource.inventorypdb.URL
26-
value: "jdbc:oracle:thin:@%INVENTORY_PDB_NAME%_tp?TNS_ADMIN=/msdataworkshop/creds"
27-
- name: inventoryqueuename
28-
value: "inventoryqueue"
29-
- name: OCI_REGION
30-
value: "%OCI_REGION%"
31-
- name: VAULT_SECRET_OCID
32-
value: "%VAULT_SECRET_OCID%"
33-
- name: dbpassword
34-
valueFrom:
35-
secretKeyRef:
36-
name: dbuser
37-
key: dbpassword
38-
optional: true #not needed/used if using VAULT_SECRET_OCID exists
23+
- name: javax.sql.DataSource.postgresDataSource.dataSource.user
24+
value: "postgresadmin"
25+
- name: javax.sql.DataSource.postgresDataSource.dataSource.URL
26+
value: "jdbc:postgresql://postgres.msdataworkshop:5432/postgresdb"
27+
- name: javax.sql.DataSource.postgresDataSource.dataSource.password
28+
value: "admin123"
29+
# - name: javax.sql.DataSource.postgresDataSource.dataSource.password.usethis
30+
# valueFrom:
31+
# secretKeyRef:
32+
# name: dbuser
33+
# key: dbpassword
3934
volumeMounts:
4035
- name: creds
4136
mountPath: /msdataworkshop/creds

grabdish/inventory-postgres-kafka/inventory-service.yaml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
1-
2-
##
31
## Copyright (c) 2021 Oracle and/or its affiliates.
42
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
53
apiVersion: v1
64
kind: Service
75
metadata:
8-
name: supplier # note that this is called supplier service as the inventory implementation services as inventory and supplier
6+
name: inventory # for various including setting of failures and eventually inventory adjustment calls, etc.
97
labels:
108
app: inventory
119
spec:

grabdish/inventory-postgres-kafka/src/main/java/io/helidon/data/examples/KafkaPostgresOrderEventConsumer.java

Lines changed: 21 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,9 @@ public class KafkaPostgresOrderEventConsumer implements Runnable {
2727
final static String inventoryTopicName = "inventory.topic";
2828
KafkaPostgressInventoryResource inventoryResource;
2929

30-
public KafkaPostgresOrderEventConsumer(KafkaPostgressInventoryResource inventoryResource) {
30+
public KafkaPostgresOrderEventConsumer(KafkaPostgressInventoryResource inventoryResource) throws SQLException {
3131
this.inventoryResource = inventoryResource;
32+
setupDB(inventoryResource.postgresDataSource.getConnection());
3233
}
3334

3435
@Override
@@ -69,9 +70,7 @@ public void listenForOrderEvents() {
6970
Order order = JsonUtils.read(txt, Order.class);
7071
System.out.print(" orderid:" + order.getOrderid());
7172
System.out.print(" itemid:" + order.getItemid());
72-
if (inventoryResource.crashAfterOrderMessageReceived) System.exit(-1);
7373
updateDataAndSendEventOnInventory(order.getOrderid(), order.getItemid());
74-
if (inventoryResource.crashAfterOrderMessageProcessed) System.exit(-1);
7574
} catch (Exception ex) {
7675
System.out.printf("message did not contain order");
7776
ex.printStackTrace();
@@ -81,11 +80,12 @@ public void listenForOrderEvents() {
8180
}
8281

8382
private void updateDataAndSendEventOnInventory( String orderid, String itemid) throws Exception {
83+
if (inventoryResource.crashAfterOrderMessageReceived) System.exit(-1);
8484
String inventorylocation = evaluateInventory(itemid);
8585
Inventory inventory = new Inventory(orderid, itemid, inventorylocation, "beer"); //static suggestiveSale - represents an additional service/event
8686
String jsonString = JsonUtils.writeValueAsString(inventory);
8787
System.out.println("send inventory status message... jsonString:" + jsonString );
88-
System.out.println("sendInsertAndSendOrderMessage.........");
88+
if (inventoryResource.crashAfterOrderMessageProcessed) System.exit(-1);
8989
String topicName = inventoryTopicName;
9090
Properties props = new Properties();
9191
props.put("bootstrap.servers", "kafka-service:9092");
@@ -109,21 +109,23 @@ private void updateDataAndSendEventOnInventory( String orderid, String itemid) t
109109
private String evaluateInventory(String id) {
110110
System.out.println("KafkaPostgresOrderEventConsumer postgresDataSource:" + inventoryResource.postgresDataSource);
111111
System.out.println("KafkaPostgresOrderEventConsumer evaluateInventory for inventoryid:" + id);
112-
String DECREMENT_BY_ID =
113-
"update inventory set inventorycount = inventorycount - 1 where inventoryid = ? and inventorycount > 0 returning inventorylocation into ?";
114-
// try (CallableStatement st = inventoryResource.postgresDataSource.getConnection().prepareCall(DECREMENT_BY_ID)) {
115112
try (PreparedStatement st = inventoryResource.postgresDataSource.getConnection().prepareStatement(
116113
"select inventorycount, inventorylocation from inventory where inventoryid = ?"
117114
)) {
118115
st.setString(1, id);
119-
// st.re.registerOutParameter(2, Types.VARCHAR);
120116
ResultSet rs = st.executeQuery();
121117
rs.next();
122118
int inventoryCount = rs.getInt(1);
123119
String inventorylocation = rs.getString(2);
124120
rs.close();
125121
System.out.println("InventoryServiceOrderEventConsumer.updateDataAndSendEventOnInventory id {" + id + "} location {" + inventorylocation + "} inventoryCount:" + inventoryCount);
126122
if (inventoryCount > 0) {
123+
PreparedStatement decrementPS = inventoryResource.postgresDataSource.getConnection().prepareStatement(
124+
"update inventory set inventorycount = ? where inventoryid = ?");
125+
decrementPS.setInt(1, inventoryCount - 1);
126+
decrementPS.setString(2, id);
127+
decrementPS.execute();
128+
System.out.println("InventoryServiceOrderEventConsumer.updateDataAndSendEventOnInventory reduced inventory count to:" + (inventoryCount - 1));
127129
return inventorylocation;
128130
} else {
129131
return "inventorydoesnotexist";
@@ -134,17 +136,22 @@ private String evaluateInventory(String id) {
134136
return "unable to find inventory status";
135137
}
136138

139+
private void setupDB(Connection connection) throws SQLException {
140+
createInventoryTable(connection);
141+
populateInventoryTable(connection);
142+
}
143+
137144
private void createInventoryTable(Connection connection) throws SQLException {
138-
System.out.println("KafkaPostgresOrderEventConsumer createInventoryTable");
145+
System.out.println("KafkaPostgresOrderEventConsumer createInventoryTable IF NOT EXISTS");
139146
connection.prepareStatement(
140-
"create table inventory ( inventoryid varchar(16) PRIMARY KEY NOT NULL, inventorylocation varchar(32), inventorycount integer CONSTRAINT positive_inventory CHECK (inventorycount >= 0) )").execute();
147+
"CREATE TABLE IF NOT EXISTS inventory ( inventoryid varchar(16) PRIMARY KEY NOT NULL, inventorylocation varchar(32), inventorycount integer CONSTRAINT positive_inventory CHECK (inventorycount >= 0) )").execute();
141148
}
142149

143150
private void populateInventoryTable(Connection connection) throws SQLException {
144-
System.out.println("KafkaPostgresOrderEventConsumer populateInventoryTable");
145-
connection.prepareStatement("insert into inventory values ('sushi', '1468 WEBSTER ST,San Francisco,CA', 0)").execute();
146-
connection.prepareStatement("insert into inventory values ('pizza', '1469 WEBSTER ST,San Francisco,CA', 0)").execute();
147-
connection.prepareStatement("insert into inventory values ('burger', '1470 WEBSTER ST,San Francisco,CA', 0)").execute();
151+
System.out.println("KafkaPostgresOrderEventConsumer populateInventoryTable if not populated");
152+
connection.prepareStatement("insert into inventory values ('sushi', '1468 WEBSTER ST,San Francisco,CA', 0) ON CONFLICT DO NOTHING").execute();
153+
connection.prepareStatement("insert into inventory values ('pizza', '1469 WEBSTER ST,San Francisco,CA', 0) ON CONFLICT DO NOTHING").execute();
154+
connection.prepareStatement("insert into inventory values ('burger', '1470 WEBSTER ST,San Francisco,CA', 0) ON CONFLICT DO NOTHING").execute();
148155
}
149156

150157

grabdish/inventory-postgres-kafka/src/main/java/io/helidon/data/examples/KafkaPostgressInventoryResource.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ public void init(@Observes @Initialized(ApplicationScoped.class) Object init) th
3737
listenForMessages();
3838
}
3939

40-
public Response listenForMessages() {
40+
public Response listenForMessages() throws SQLException {
4141
new Thread(new KafkaPostgresOrderEventConsumer(this)).start();
4242
final Response returnValue = Response.ok()
4343
.entity("now listening for messages...")
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
## Copyright (c) 2021 Oracle and/or its affiliates.
2+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
3+
apiVersion: v1
4+
kind: Service
5+
metadata:
6+
name: supplier # note that this is called supplier service as the inventory implementation services as inventory and supplier
7+
labels:
8+
app: inventory
9+
spec:
10+
type: NodePort
11+
ports:
12+
- port: 8080
13+
name: http
14+
selector:
15+
app: inventory
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/bin/bash
2+
## Copyright (c) 2021 Oracle and/or its affiliates.
3+
## Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
4+
5+
echo install kafka...
6+
./install-kafka.sh
7+
8+
echo install mongodb...
9+
./install-mongodb.sh
10+
11+
echo install postgres
12+
./install-postgres.sh

grabdish/mongodb-kafka-postgres/install-mongodb.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,4 @@
55
kubectl create -f mongodata-persistentvolumeclaim.yaml -n msdataworkshop
66
kubectl create -f mongodb-deployment.yaml -n msdataworkshop
77
kubectl create -f mongodb-service.yaml -n msdataworkshop
8-
kubectl expose deployment mongodb --type=LoadBalancer -n msdataworkshop
8+
#kubectl expose deployment mongodb --type=LoadBalancer -n msdataworkshop # remove as we have service

grabdish/mongodb-kafka-postgres/install-postgres.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,5 @@ kubectl create -f postgres-configmap.yaml -n msdataworkshop
66
kubectl create -f postgres-storage.yaml -n msdataworkshop
77
kubectl create -f postgres-deployment.yaml -n msdataworkshop
88
kubectl create -f postgres-service.yaml -n msdataworkshop
9-
kubectl get svc postgres
9+
#kubectl get svc postgres -n msdataworkshop
1010

0 commit comments

Comments
 (0)