Add nix shell informations

README.md

@@ -2,19 +2,60 @@
 
 Local dev environment for the COOPGO Technical Platform, using [Tilt](https://tilt.dev) and a local Kubernetes cluster (such as [Rancher Desktop](https://rancherdesktop.io/), K3D, ...)
 
-Tested with Rancher Desktop.
+Tested with Rancher Desktop and K3D.
 
 ## Quickstart
 
-1. Install Rancher Dektop or K3D or any other local Kubernetes. Make your Kubernetes context point to this cluster
+### Install local cluster
 
-2. [Install Tilt](https://docs.tilt.dev/install.html)
+Install Rancher Dektop or K3D or any other local Kubernetes. Make your Kubernetes context point to this cluster
 
-3. Create a .env file and configure environment
+Later, we will be using port 80 to access the cluster Ingress controller. On Linux, you must probably allow this.
+
+```
+sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
+```
+
+To preserve this change across reboots as a custom kernel parameter setting, add the same command inside your /etc/sysctl.conf file (or a dedicated /etc/sysctl.d/xxxxx.conf).
+
+#### Rancher Desktop
+
+See installation instructions here : <https://docs.rancherdesktop.io/getting-started/installation>
+
+#### K3D
+
+Install K3D :
+
+```
+curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
+```
+
+Create config for the internal K3D registry to mirror your default docker images registry, for example in `/tmp/k3d-registry.yaml` :
+
+```/tmp/k3d.registry
+mirrors:
+  docker.io:
+    endpoint:
+      - "git.coopgo.io"
+```
+
+Create cluster :
+
+```
+k3d cluster create devcluster -p "80:80@loadbalancer" --registry-create devregistry:5000 --registry-config /tmp/k3d-registry.yaml
+```
+
+### Install Tilt
+
+[Follow instructions on Tilt documentation](https://docs.tilt.dev/install.html)
+
+### Set the environment
+
+Create a .env file and configure environment
 
 For example, to set the source code directories (see default values in Tiltfile) :
 
-```
+```.env
 PARCOURSMOB_DIR="../../coopgo-apps/parcoursmob"
 MOBILITY_ACCOUNTS_DIR="../../coopgo-platform/mobility-accounts"
 AGENDA_DIR="../../coopgo-platform/agenda"
@@ -24,29 +65,47 @@ GROUPS_MANAGEMENT_DIR="../../coopgo-platform/groups-management"
 
 To avoid developing on another cluster than the one dedicated for dev, we whitelist the correct context. By default, the allowed context is 'rancher-desktop'. To change this, set ALLOWED_K8S_CONTEXT to the correct context name in .env :
 
-```
+```.env
 ALLOWED_K8S_CONTEXT="k3d-devcluster"
 ```
 
 You might also want to set the default container images used if/while your local source code folder doesn't exist.
 
-```
-PARCOURSMOB_IMAGE=my.own.registry/coopgo-platform/parcoursmob@v1.2
+```.env
+PARCOURSMOB_IMAGE=coopgo-apps/parcoursmob@v1.2
 MOBILITY_ACCOUNTS_IMAGE=my.own.registry/coopgo-platform/mobility-accounts
 ...
 ```
 
-4. Add `tilt-dev.svc.cluster.local parcoursmob.tilt-dev.svc.cluster.local mobility-accounts.tilt-dev.svc.cluster.local` to resolve `127.0.0.1` in your hosts config (`/etc/hosts` on Linux)
+#### K3D specific setup
 
-5. Run `tilt up`
+If you're using K3D, remove the registry part in the images (we'll use K3D local registry instead). It should look something like this :
 
-6. After some time (you can see the services loading -or failing, if there is an error- through Tilt UI), PARCOURSMOB should be available on http://parcoursmob.tilt-dev.svc.cluster.local
+```.env
+PARCOURSMOB_IMAGE=coopgo-apps/parcoursmob 
+MOBILITY_ACCOUNTS_IMAGE=coopgo-platform/mobility-accounts
+...
+```
 
-To have this work, make sure port 80 is unprivileged if you're using Linux. This is described in Rancher Desktop documentation. If you're using something else to run Kubernetes, make sure you have the right port forwarding and an Ingress controller on your cluster to target port 80 on localhost.
+Then, set the K8S config to the dedicated overlay in k8s_config :
+
+```.env
+K8S_CONFIG="k8s_config/k3d"
+```
+
+### Set local hosts config
+
+Add `tilt-dev.svc.cluster.local parcoursmob.tilt-dev.svc.cluster.local mobility-accounts.tilt-dev.svc.cluster.local` to resolve `127.0.0.1` in your hosts config (`/etc/hosts` on Linux)
+
+### Run the dev environment and start developing
+
+Run `tilt up`
+
+After some time (you can see the services loading -or failing, if there is an error- through Tilt UI), PARCOURSMOB should be available on <http://parcoursmob.tilt-dev.svc.cluster.local>
 
 Once you can acess PARCOURSMOB UI, you can connect with the initially created user :
 
-- username: admin@parcoursmob.fr
+- username: <admin@parcoursmob.fr>
 - password: parcoursmob
 
 Now, you're ready to code !
@@ -56,16 +115,16 @@ Now, you're ready to code !
 Set up PARCOURSMOB along with the following dependencies :
 
 - COOPGO microservices :
-    - Mobility Accounts
-    - Groups Management
-    - Fleets
-    - Agenda
-    - Carpool service (coming soon)
+  - Mobility Accounts
+  - Groups Management
+  - Fleets
+  - Agenda
+  - Carpool service (coming soon)
 - Other software Dependencies :
-    - MongoDB (Next step, deploy with PostgreSQL as both should be supported)
-    - Etcd for caching values
-    - [Minio](https://min.io) to store files
-    - [smtp4dev](https://github.com/rnwood/smtp4dev) to test outgoing emails
+  - MongoDB (Next step, deploy with PostgreSQL as both should be supported)
+  - Etcd for caching values
+  - [Minio](https://min.io) to store files
+  - [smtp4dev](https://github.com/rnwood/smtp4dev) to test outgoing emails
 
 Tilt handles livereloading Kubernetes pods when something changes in the code. You just have to checkout your code in the directory you've specified in .env (see quickstart), start developing on exsting microservices, and see the changes.
 
@@ -73,5 +132,20 @@ Tilt handles livereloading Kubernetes pods when something changes in the code. Y
 
 Access smtp4dev to check outgoing emails :
 
-1. Forward smtp4dev service locally : kubectl port-forward svc/smtp4dev 9090:80
-2. Access UI : http://localhost:9090/
+By default, smtp4dev HTTP port is redirected to localhost:12345. You can change this with the dedicated environment variable in `.env`
+
+```.env
+SMTP4DEV_LOCAL_PORT=12345
+```
+
+## Use Nix shell to start easily
+
+You can use the "nix shell" (<https://nixos.org/>) utility to set your environment without having to install every dependency individually.
+
+Install Nix, run `nix-shell`, and benefit from a shell providing :
+
+- Tilt command
+- K3D
+- Protocol buffer dependencies (protoc, protoc-gen-go, protobuf) needed to build gRPC protocol buffer definitions in the Golang microservices with the COOPGO Platform
+
+More to come to provide a full dev environment dedicated to the COOPGO Technical Platform

Tiltfile

@@ -5,28 +5,21 @@ dotenv()
 
 allowed_k8s_context = os.environ.get('ALLOWED_K8S_CONTEXT', 'rancher-desktop')
 
-# parcoursmob_dir = os.path.realpath(os.environ.get('PARCOURSMOB_DIR', 'src/parcoursmob'))
-# parcoursmob_theme_dir = os.path.realpath(os.environ.get('PARCOURSMOB_THEME_DIR', 'src/parcoursmob/themes/default'))
-# mobility_accounts_dir = os.path.realpath(os.environ.get('MOBILITY_ACCOUNTS_DIR', 'src/mobility-accounts'))
-# agenda_dir = os.path.realpath(os.environ.get('AGENDA_DIR', 'src/agenda'))
-# fleets_dir = os.path.realpath(os.environ.get('FLEETS_DIR', 'src/fleets'))
-# groups_management_dir = os.path.realpath(os.environ.get('GROUPS_MANAGEMENT_DIR', 'src/groups-management'))
-
 parcoursmob_dir = os.environ.get('PARCOURSMOB_DIR', 'src/parcoursmob')
 parcoursmob_theme_dir = os.environ.get('PARCOURSMOB_THEME_DIR', 'src/parcoursmob/themes/default')
 mobility_accounts_dir = os.environ.get('MOBILITY_ACCOUNTS_DIR', 'src/mobility-accounts')
 agenda_dir = os.environ.get('AGENDA_DIR', 'src/agenda')
 fleets_dir = os.environ.get('FLEETS_DIR', 'src/fleets')
 groups_management_dir = os.environ.get('GROUPS_MANAGEMENT_DIR', 'src/groups-management')
-diags_dir = os.environ.get('DIAGS_DIR', 'src/diags')
 
 parcoursmob_image = os.environ.get('PARCOURSMOB_IMAGE', 'git.coopgo.io/coopgo-apps/parcoursmob')
 mobility_accounts_image = os.environ.get('MOBILITY_ACCOUNTS_IMAGE', 'git.coopgo.io/coopgo-platform/mobility-accounts')
 agenda_image = os.environ.get('AGENDA_IMAGE', 'git.coopgo.io/coopgo-platform/agenda')
 fleets_image = os.environ.get('FLEETS_IMAGE', 'git.coopgo.io/coopgo-platform/fleets')
 groups_management_image = os.environ.get('GROUPS_MANAGEMENT_IMAGE', 'git.coopgo.io/coopgo-platform/groups-management')
-diags_image = os.environ.get('DIAGS_IMAGE', 'git.coopgo.io/coopgo-platform/diags')
 
+k8s_config = os.environ.get('K8S_CONFIG', 'k8s_config/dev')
+smtp4dev_local_port = os.environ.get("SMTP4DEV_LOCAL_PORT", '12345')
 
 allow_k8s_contexts(allowed_k8s_context)
 
@@ -74,11 +67,6 @@ if os.path.exists(groups_management_dir):
     print("Directory : {dir}".format(dir=groups_management_dir))
     deploy_go('groups-management', groups_management_dir, groups_management_image)
 
-if os.path.exists(diags_dir):
-    diags_dir =os.path.realpath(diags_dir)
-    print("DIAGS")
-    print("Directory : {dir}".format(dir=diags_dir))
-    deploy_go('diags', diags_dir, diags_image)
-
-k8s_yaml(kustomize('k8s_config/dev'))
+k8s_yaml(kustomize(k8s_config))
+k8s_resource(workload='smtp4dev', port_forwards=smtp4dev_local_port+":80")
 

k8s_config/dev/diags/config.yaml

@@ -1,16 +0,0 @@
-
-dev_env: true
-
-storage:
-  db:
-    type: mongodb
-    mongodb:
-      host: mongodb
-      port: 27017
-      db_name: coopgo_platform
-      collections:
-        diags: diags
-
-services:
-  grpc:
-    port: 8080

k8s_config/dev/diags/deployment.yaml

@@ -1,31 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: diags
-spec:
-  selector:
-    matchLabels:
-      app: diags
-  template:
-    metadata:
-      labels:
-        app: diags
-    spec:
-      containers:
-      - name: diags
-        image: git.coopgo.io/coopgo-platform/diags:dev
-        ports:
-        - name: grpc
-          containerPort: 8080
-          protocol: TCP
-        volumeMounts:
-        - name: diags-config-volume 
-          mountPath: /config.yaml
-          subPath: config.yaml
-      volumes:
-      - name: diags-config-volume 
-        configMap:
-          name: diags-config
-          items:
-            - key: config.yaml
-              path: config.yaml

k8s_config/dev/diags/kustomization.yaml

@@ -1,8 +0,0 @@
-resources:
-- deployment.yaml 
-- service.yaml 
-
-configMapGenerator:
-- name: diags-config
-  files:
-    - config.yaml

k8s_config/dev/diags/service.yaml

@@ -1,11 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: diags
-spec:
-  selector:
-    app: diags
-  ports:
-  - port: 8080
-    name: grpc
-    targetPort: 8080

k8s_config/dev/kustomization.yaml

@@ -6,7 +6,6 @@ resources:
 - ./fleets/
 - ./groups-management/
 - ./mobility-accounts/
-- ./diags/
 
 # Apps
 - ./parcoursmob/

k8s_config/dev/mongodb/create_db.js

@@ -24,7 +24,7 @@ db.users.insert({
         "first_name" : "Admin",
         "last_name" : "Parcoursmob",
         "phone_number" : "+33600000000",
-        "email" : "noreply@parcoursmob.fr",
+        "email" : "admin@parcoursmob.fr",
         "groups" : [
             "483280d0-db2d-4f06-b361-02e4be5012d2",
             "483280d0-db2d-4f06-b361-02e4be5012d2:admin",
@@ -50,8 +50,7 @@ db.groups.insert({
             "vehicles" : true,
             "support" : true,
             "beneficiaries" : true,
-            "journeys" : true,
-            "diags" : true
+            "journeys" : true
         },
         "name" : "COOPGO"
     }

k8s_config/dev/parcoursmob/config.yaml

@@ -30,8 +30,6 @@ services:
       dial: fleets:8080
     agenda:
       dial: agenda:8080
-    diags:
-      dial: diags:8080
   navitia:
     api_key: abcd
 
@@ -87,6 +85,11 @@ modules:
         - work_contract
   agenda:
     enabled: true
+    calendars:
+      global:
+        enabled: true
+      organizations:
+        enabled: true
     events_types:
       - name: Accompagnements individuels
       - name: Permis accéléré
@@ -99,7 +102,6 @@ modules:
       - name: Auto-école sociale (classique)
       - name: Information collective
       - name: Autre
-    documents_types:
-      - notice
-      - pieces
-      - other
+
+#  knowledge:
+#    enabled: true

k8s_config/dev/parcoursmob/deployment.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: parcoursmob
-        image: git.coopgo.io/coopgo-apps/parcoursmob:dev
+        image: git.coopgo.io/coopgo-apps/parcoursmob
         ports:
         - containerPort: 80
         volumeMounts:

k8s_config/k3d/kustomization.yaml

@@ -0,0 +1,12 @@
+resources:
+  - ../dev/
+
+images:
+  - name: git.coopgo.io/coopgo-apps/parcoursmob
+    newName: "coopgo-apps/parcoursmob"
+  - name: git.coopgo.io/coopgo-platform/mobility-accounts
+    newName: "coopgo-platform/mobility-accounts"
+  #  - name: git.coopgo.io/coopgo-platform/fleets
+  #    newName: "coopgo-platform/fleets"
+  - name: git.coopgo.io/coopgo-platform/agenda
+    newName: "coopgo-platform/agenda"

shell.nix

@@ -0,0 +1,16 @@
+let
+  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/nixos-24.05";
+  pkgs = import nixpkgs { config = {}; overlays = []; };
+in
+
+pkgs.mkShellNoCC {
+  packages = with pkgs; [
+    tilt
+    k3d
+    neovim
+    kubectl
+    protobuf
+    protoc-gen-go
+    protoc-gen-go-grpc
+  ];
+}