add diags services

README.md

@@ -2,60 +2,19 @@
 
 Local dev environment for the COOPGO Technical Platform, using [Tilt](https://tilt.dev) and a local Kubernetes cluster (such as [Rancher Desktop](https://rancherdesktop.io/), K3D, ...)
 
-Tested with Rancher Desktop and K3D.
+Tested with Rancher Desktop.
 
 ## Quickstart
 
-### Install local cluster
+1. Install Rancher Dektop or K3D or any other local Kubernetes. Make your Kubernetes context point to this cluster
 
-Install Rancher Dektop or K3D or any other local Kubernetes. Make your Kubernetes context point to this cluster
+2. [Install Tilt](https://docs.tilt.dev/install.html)
 
-Later, we will be using port 80 to access the cluster Ingress controller. On Linux, you must probably allow this.
+3. Create a .env file and configure environment
-
-```
-$ sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
-```
-
-To preserve this change across reboots as a custom kernel parameter setting, add the same command inside your /etc/sysctl.conf file (or a dedicated /etc/sysctl.d/xxxxx.conf).
-
-#### Rancher Desktop
-
-See installation instructions here : https://docs.rancherdesktop.io/getting-started/installation
-
-#### K3D 
-
-Install K3D : 
-
-```
-$ curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
-```
-
-Create config for the internal K3D registry to mirror your default docker images registry, for example in `/tmp/k3d-registry.yaml` :
-
-```/tmp/k3d.registry
-mirrors:
-  docker.io:
-    endpoint:
-      - "git.coopgo.io"
-```
-
-Create cluster :
-
-``` 
-$ k3d cluster create devcluster -p "80:80@loadbalancer" --registry-create devregistry:5000 --registry-config /tmp/k3d-registry.yaml
-```
-
-### Install Tilt 
-
-[Follow instructions on Tilt documentation](https://docs.tilt.dev/install.html)
-
-### Set the environment
-
-Create a .env file and configure environment
 
 For example, to set the source code directories (see default values in Tiltfile) :
 
-```.env
+```
 PARCOURSMOB_DIR="../../coopgo-apps/parcoursmob"
 MOBILITY_ACCOUNTS_DIR="../../coopgo-platform/mobility-accounts"
 AGENDA_DIR="../../coopgo-platform/agenda"
@@ -65,43 +24,25 @@ GROUPS_MANAGEMENT_DIR="../../coopgo-platform/groups-management"
 
 To avoid developing on another cluster than the one dedicated for dev, we whitelist the correct context. By default, the allowed context is 'rancher-desktop'. To change this, set ALLOWED_K8S_CONTEXT to the correct context name in .env :
 
-```.env
+```
 ALLOWED_K8S_CONTEXT="k3d-devcluster"
 ```
 
 You might also want to set the default container images used if/while your local source code folder doesn't exist.
 
-```.env
+```
-PARCOURSMOB_IMAGE=coopgo-apps/parcoursmob@v1.2
+PARCOURSMOB_IMAGE=my.own.registry/coopgo-platform/parcoursmob@v1.2
 MOBILITY_ACCOUNTS_IMAGE=my.own.registry/coopgo-platform/mobility-accounts
 ...
 ```
 
-#### K3D specific setup 
+4. Add `tilt-dev.svc.cluster.local parcoursmob.tilt-dev.svc.cluster.local mobility-accounts.tilt-dev.svc.cluster.local` to resolve `127.0.0.1` in your hosts config (`/etc/hosts` on Linux)
 
-If you're using K3D, remove the registry part in the images (we'll use K3D local registry instead). It should look something like this :
+5. Run `tilt up`
 
-```.env
+6. After some time (you can see the services loading -or failing, if there is an error- through Tilt UI), PARCOURSMOB should be available on http://parcoursmob.tilt-dev.svc.cluster.local
-PARCOURSMOB_IMAGE=coopgo-apps/parcoursmob 
-MOBILITY_ACCOUNTS_IMAGE=coopgo-platform/mobility-accounts
-...
-```
 
-Then, set the K8S config to the dedicated overlay in k8s_config :
+To have this work, make sure port 80 is unprivileged if you're using Linux. This is described in Rancher Desktop documentation. If you're using something else to run Kubernetes, make sure you have the right port forwarding and an Ingress controller on your cluster to target port 80 on localhost.
-
-```.env 
-K8S_CONFIG="k8s_config/k3d"
-```
-
-### Set local hosts config 
-
-Add `tilt-dev.svc.cluster.local parcoursmob.tilt-dev.svc.cluster.local mobility-accounts.tilt-dev.svc.cluster.local` to resolve `127.0.0.1` in your hosts config (`/etc/hosts` on Linux)
-
-### Run the dev environment and start developing 
-
-Run `tilt up`
-
-After some time (you can see the services loading -or failing, if there is an error- through Tilt UI), PARCOURSMOB should be available on http://parcoursmob.tilt-dev.svc.cluster.local
 
 Once you can acess PARCOURSMOB UI, you can connect with the initially created user :
 
@@ -132,8 +73,5 @@ Tilt handles livereloading Kubernetes pods when something changes in the code. Y
 
 Access smtp4dev to check outgoing emails :
 
-By default, smtp4dev HTTP port is redirected to localhost:12345. You can change this with the dedicated environment variable in `.env`
+1. Forward smtp4dev service locally : kubectl port-forward svc/smtp4dev 9090:80
-
+2. Access UI : http://localhost:9090/
-```.env
-SMTP4DEV_LOCAL_PORT=12345
-```

Tiltfile

@@ -5,21 +5,28 @@ dotenv()
 
 allowed_k8s_context = os.environ.get('ALLOWED_K8S_CONTEXT', 'rancher-desktop')
 
+# parcoursmob_dir = os.path.realpath(os.environ.get('PARCOURSMOB_DIR', 'src/parcoursmob'))
+# parcoursmob_theme_dir = os.path.realpath(os.environ.get('PARCOURSMOB_THEME_DIR', 'src/parcoursmob/themes/default'))
+# mobility_accounts_dir = os.path.realpath(os.environ.get('MOBILITY_ACCOUNTS_DIR', 'src/mobility-accounts'))
+# agenda_dir = os.path.realpath(os.environ.get('AGENDA_DIR', 'src/agenda'))
+# fleets_dir = os.path.realpath(os.environ.get('FLEETS_DIR', 'src/fleets'))
+# groups_management_dir = os.path.realpath(os.environ.get('GROUPS_MANAGEMENT_DIR', 'src/groups-management'))
+
 parcoursmob_dir = os.environ.get('PARCOURSMOB_DIR', 'src/parcoursmob')
 parcoursmob_theme_dir = os.environ.get('PARCOURSMOB_THEME_DIR', 'src/parcoursmob/themes/default')
 mobility_accounts_dir = os.environ.get('MOBILITY_ACCOUNTS_DIR', 'src/mobility-accounts')
 agenda_dir = os.environ.get('AGENDA_DIR', 'src/agenda')
 fleets_dir = os.environ.get('FLEETS_DIR', 'src/fleets')
 groups_management_dir = os.environ.get('GROUPS_MANAGEMENT_DIR', 'src/groups-management')
+diags_dir = os.environ.get('DIAGS_DIR', 'src/diags')
 
 parcoursmob_image = os.environ.get('PARCOURSMOB_IMAGE', 'git.coopgo.io/coopgo-apps/parcoursmob')
 mobility_accounts_image = os.environ.get('MOBILITY_ACCOUNTS_IMAGE', 'git.coopgo.io/coopgo-platform/mobility-accounts')
 agenda_image = os.environ.get('AGENDA_IMAGE', 'git.coopgo.io/coopgo-platform/agenda')
 fleets_image = os.environ.get('FLEETS_IMAGE', 'git.coopgo.io/coopgo-platform/fleets')
 groups_management_image = os.environ.get('GROUPS_MANAGEMENT_IMAGE', 'git.coopgo.io/coopgo-platform/groups-management')
+diags_image = os.environ.get('DIAGS_IMAGE', 'git.coopgo.io/coopgo-platform/diags')
 
-k8s_config = os.environ.get('K8S_CONFIG', 'k8s_config/dev')
-smtp4dev_local_port = os.environ.get("SMTP4DEV_LOCAL_PORT", '12345')
 
 allow_k8s_contexts(allowed_k8s_context)
 
@@ -67,6 +74,11 @@ if os.path.exists(groups_management_dir):
     print("Directory : {dir}".format(dir=groups_management_dir))
     deploy_go('groups-management', groups_management_dir, groups_management_image)
 
-k8s_yaml(kustomize(k8s_config))
+if os.path.exists(diags_dir):
-k8s_resource(workload='smtp4dev', port_forwards=smtp4dev_local_port+":80")
+    diags_dir =os.path.realpath(diags_dir)
+    print("DIAGS")
+    print("Directory : {dir}".format(dir=diags_dir))
+    deploy_go('diags', diags_dir, diags_image)
+
+k8s_yaml(kustomize('k8s_config/dev'))
 

k8s_config/dev/diags/config.yaml

@@ -0,0 +1,16 @@
+
+dev_env: true
+
+storage:
+  db:
+    type: mongodb
+    mongodb:
+      host: mongodb
+      port: 27017
+      db_name: coopgo_platform
+      collections:
+        events: diags
+
+services:
+  grpc:
+    port: 8080

k8s_config/dev/diags/deployment.yaml

@@ -0,0 +1,31 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: diags
+spec:
+  selector:
+    matchLabels:
+      app: diags
+  template:
+    metadata:
+      labels:
+        app: diags
+    spec:
+      containers:
+      - name: diags
+        image: git.coopgo.io/coopgo-platform/diags:dev
+        ports:
+        - name: grpc
+          containerPort: 8080
+          protocol: TCP
+        volumeMounts:
+        - name: diags-config-volume 
+          mountPath: /config.yaml
+          subPath: config.yaml
+      volumes:
+      - name: diags-config-volume 
+        configMap:
+          name: diags-config
+          items:
+            - key: config.yaml
+              path: config.yaml

k8s_config/dev/diags/kustomization.yaml

@@ -0,0 +1,8 @@
+resources:
+- deployment.yaml 
+- service.yaml 
+
+configMapGenerator:
+- name: diags-config
+  files:
+    - config.yaml

k8s_config/dev/diags/service.yaml

@@ -0,0 +1,11 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: diags
+spec:
+  selector:
+    app: diags
+  ports:
+  - port: 8080
+    name: grpc
+    targetPort: 8080

k8s_config/dev/kustomization.yaml

@@ -6,6 +6,7 @@ resources:
 - ./fleets/
 - ./groups-management/
 - ./mobility-accounts/
+- ./diags/
 
 # Apps
 - ./parcoursmob/

k8s_config/dev/mongodb/create_db.js

@@ -24,7 +24,7 @@ db.users.insert({
         "first_name" : "Admin",
         "last_name" : "Parcoursmob",
         "phone_number" : "+33600000000",
-        "email" : "admin@parcoursmob.fr",
+        "email" : "noreply@parcoursmob.fr",
         "groups" : [
             "483280d0-db2d-4f06-b361-02e4be5012d2",
             "483280d0-db2d-4f06-b361-02e4be5012d2:admin",
@@ -54,4 +54,4 @@ db.groups.insert({
         },
         "name" : "COOPGO"
     }
-})
+})

k8s_config/dev/parcoursmob/config.yaml

@@ -30,6 +30,8 @@ services:
       dial: fleets:8080
     agenda:
       dial: agenda:8080
+    diags:
+      dial: diags:8080
   navitia:
     api_key: abcd
 
@@ -97,4 +99,7 @@ modules:
       - name: Auto-école sociale (classique)
       - name: Information collective
       - name: Autre
-
+    documents_types:
+      - notice
+      - pieces
+      - other

k8s_config/dev/parcoursmob/deployment.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: parcoursmob
-        image: git.coopgo.io/coopgo-apps/parcoursmob
+        image: git.coopgo.io/coopgo-apps/parcoursmob:dev
         ports:
         - containerPort: 80
         volumeMounts:

k8s_config/k3d/kustomization.yaml

@@ -1,12 +0,0 @@
-resources: 
-  - ../dev/ 
-
-images:
-  - name: git.coopgo.io/coopgo-apps/parcoursmob
-    newName: "coopgo-apps/parcoursmob"
-  - name: git.coopgo.io/coopgo-platform/mobility-accounts
-    newName: "coopgo-platform/mobility-accounts"
-  - name: git.coopgo.io/coopgo-platform/fleets
-    newName: "coopgo-platform/fleets"
-  - name: git.coopgo.io/coopgo-platform/agenda
-    newName: "coopgo-platform/agenda"