redirect smtp4dev port to localhost

README.md

@@ -2,19 +2,60 @@
 
 Local dev environment for the COOPGO Technical Platform, using [Tilt](https://tilt.dev) and a local Kubernetes cluster (such as [Rancher Desktop](https://rancherdesktop.io/), K3D, ...)
 
-Tested with Rancher Desktop.
+Tested with Rancher Desktop and K3D.
 
 ## Quickstart
 
-1. Install Rancher Dektop or K3D or any other local Kubernetes. Make your Kubernetes context point to this cluster
+### Install local cluster
 
-2. [Install Tilt](https://docs.tilt.dev/install.html)
+Install Rancher Dektop or K3D or any other local Kubernetes. Make your Kubernetes context point to this cluster
 
-3. Create a .env file and configure environment
+Later, we will be using port 80 to access the cluster Ingress controller. On Linux, you must probably allow this.
+
+```
+$ sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
+```
+
+To preserve this change across reboots as a custom kernel parameter setting, add the same command inside your /etc/sysctl.conf file (or a dedicated /etc/sysctl.d/xxxxx.conf).
+
+#### Rancher Desktop
+
+See installation instructions here : https://docs.rancherdesktop.io/getting-started/installation
+
+#### K3D 
+
+Install K3D : 
+
+```
+$ curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
+```
+
+Create config for the internal K3D registry to mirror your default docker images registry, for example in `/tmp/k3d-registry.yaml` :
+
+```/tmp/k3d.registry
+mirrors:
+  docker.io:
+    endpoint:
+      - "git.coopgo.io"
+```
+
+Create cluster :
+
+``` 
+$ k3d cluster create devcluster -p "80:80@loadbalancer" --registry-create devregistry:5000 --registry-config /tmp/k3d-registry.yaml
+```
+
+### Install Tilt 
+
+[Follow instructions on Tilt documentation](https://docs.tilt.dev/install.html)
+
+### Set the environment
+
+Create a .env file and configure environment
 
 For example, to set the source code directories (see default values in Tiltfile) :
 
-```
+```.env
 PARCOURSMOB_DIR="../../coopgo-apps/parcoursmob"
 MOBILITY_ACCOUNTS_DIR="../../coopgo-platform/mobility-accounts"
 AGENDA_DIR="../../coopgo-platform/agenda"
@@ -24,25 +65,43 @@ GROUPS_MANAGEMENT_DIR="../../coopgo-platform/groups-management"
 
 To avoid developing on another cluster than the one dedicated for dev, we whitelist the correct context. By default, the allowed context is 'rancher-desktop'. To change this, set ALLOWED_K8S_CONTEXT to the correct context name in .env :
 
-```
+```.env
 ALLOWED_K8S_CONTEXT="k3d-devcluster"
 ```
 
 You might also want to set the default container images used if/while your local source code folder doesn't exist.
 
-```
-PARCOURSMOB_IMAGE=my.own.registry/coopgo-platform/parcoursmob@v1.2
+```.env
+PARCOURSMOB_IMAGE=coopgo-apps/parcoursmob@v1.2
 MOBILITY_ACCOUNTS_IMAGE=my.own.registry/coopgo-platform/mobility-accounts
 ...
 ```
 
-4. Add `tilt-dev.svc.cluster.local parcoursmob.tilt-dev.svc.cluster.local mobility-accounts.tilt-dev.svc.cluster.local` to resolve `127.0.0.1` in your hosts config (`/etc/hosts` on Linux)
+#### K3D specific setup 
 
-5. Run `tilt up`
+If you're using K3D, remove the registry part in the images (we'll use K3D local registry instead). It should look something like this :
 
-6. After some time (you can see the services loading -or failing, if there is an error- through Tilt UI), PARCOURSMOB should be available on http://parcoursmob.tilt-dev.svc.cluster.local
+```.env
+PARCOURSMOB_IMAGE=coopgo-apps/parcoursmob 
+MOBILITY_ACCOUNTS_IMAGE=coopgo-platform/mobility-accounts
+...
+```
 
-To have this work, make sure port 80 is unprivileged if you're using Linux. This is described in Rancher Desktop documentation. If you're using something else to run Kubernetes, make sure you have the right port forwarding and an Ingress controller on your cluster to target port 80 on localhost.
+Then, set the K8S config to the dedicated overlay in k8s_config :
+
+```.env 
+K8S_CONFIG="k8s_config/k3d"
+```
+
+### Set local hosts config 
+
+Add `tilt-dev.svc.cluster.local parcoursmob.tilt-dev.svc.cluster.local mobility-accounts.tilt-dev.svc.cluster.local` to resolve `127.0.0.1` in your hosts config (`/etc/hosts` on Linux)
+
+### Run the dev environment and start developing 
+
+Run `tilt up`
+
+After some time (you can see the services loading -or failing, if there is an error- through Tilt UI), PARCOURSMOB should be available on http://parcoursmob.tilt-dev.svc.cluster.local
 
 Once you can acess PARCOURSMOB UI, you can connect with the initially created user :
 
@@ -73,5 +132,8 @@ Tilt handles livereloading Kubernetes pods when something changes in the code. Y
 
 Access smtp4dev to check outgoing emails :
 
-1. Forward smtp4dev service locally : kubectl port-forward svc/smtp4dev 9090:80
-2. Access UI : http://localhost:9090/
+By default, smtp4dev HTTP port is redirected to localhost:12345. You can change this with the dedicated environment variable in `.env`
+
+```.env
+SMTP4DEV_LOCAL_PORT=12345
+```

Tiltfile

@@ -5,28 +5,21 @@ dotenv()
 
 allowed_k8s_context = os.environ.get('ALLOWED_K8S_CONTEXT', 'rancher-desktop')
 
-# parcoursmob_dir = os.path.realpath(os.environ.get('PARCOURSMOB_DIR', 'src/parcoursmob'))
-# parcoursmob_theme_dir = os.path.realpath(os.environ.get('PARCOURSMOB_THEME_DIR', 'src/parcoursmob/themes/default'))
-# mobility_accounts_dir = os.path.realpath(os.environ.get('MOBILITY_ACCOUNTS_DIR', 'src/mobility-accounts'))
-# agenda_dir = os.path.realpath(os.environ.get('AGENDA_DIR', 'src/agenda'))
-# fleets_dir = os.path.realpath(os.environ.get('FLEETS_DIR', 'src/fleets'))
-# groups_management_dir = os.path.realpath(os.environ.get('GROUPS_MANAGEMENT_DIR', 'src/groups-management'))
-
 parcoursmob_dir = os.environ.get('PARCOURSMOB_DIR', 'src/parcoursmob')
 parcoursmob_theme_dir = os.environ.get('PARCOURSMOB_THEME_DIR', 'src/parcoursmob/themes/default')
 mobility_accounts_dir = os.environ.get('MOBILITY_ACCOUNTS_DIR', 'src/mobility-accounts')
 agenda_dir = os.environ.get('AGENDA_DIR', 'src/agenda')
 fleets_dir = os.environ.get('FLEETS_DIR', 'src/fleets')
 groups_management_dir = os.environ.get('GROUPS_MANAGEMENT_DIR', 'src/groups-management')
-diags_dir = os.environ.get('DIAGS_DIR', 'src/diags')
 
 parcoursmob_image = os.environ.get('PARCOURSMOB_IMAGE', 'git.coopgo.io/coopgo-apps/parcoursmob')
 mobility_accounts_image = os.environ.get('MOBILITY_ACCOUNTS_IMAGE', 'git.coopgo.io/coopgo-platform/mobility-accounts')
 agenda_image = os.environ.get('AGENDA_IMAGE', 'git.coopgo.io/coopgo-platform/agenda')
 fleets_image = os.environ.get('FLEETS_IMAGE', 'git.coopgo.io/coopgo-platform/fleets')
 groups_management_image = os.environ.get('GROUPS_MANAGEMENT_IMAGE', 'git.coopgo.io/coopgo-platform/groups-management')
-diags_image = os.environ.get('DIAGS_IMAGE', 'git.coopgo.io/coopgo-platform/diags')
 
+k8s_config = os.environ.get('K8S_CONFIG', 'k8s_config/dev')
+smtp4dev_local_port = os.environ.get("SMTP4DEV_LOCAL_PORT", '12345')
 
 allow_k8s_contexts(allowed_k8s_context)
 
@@ -74,11 +67,6 @@ if os.path.exists(groups_management_dir):
     print("Directory : {dir}".format(dir=groups_management_dir))
     deploy_go('groups-management', groups_management_dir, groups_management_image)
 
-if os.path.exists(diags_dir):
-    diags_dir =os.path.realpath(diags_dir)
-    print("DIAGS")
-    print("Directory : {dir}".format(dir=diags_dir))
-    deploy_go('diags', diags_dir, diags_image)
-
-k8s_yaml(kustomize('k8s_config/dev'))
+k8s_yaml(kustomize(k8s_config))
+k8s_resource(workload='smtp4dev', port_forwards=smtp4dev_local_port+":80")
 

k8s_config/dev/diags/config.yaml

@@ -1,16 +0,0 @@
-
-dev_env: true
-
-storage:
-  db:
-    type: mongodb
-    mongodb:
-      host: mongodb
-      port: 27017
-      db_name: coopgo_platform
-      collections:
-        events: diags
-
-services:
-  grpc:
-    port: 8080

k8s_config/dev/diags/deployment.yaml

@@ -1,31 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: diags
-spec:
-  selector:
-    matchLabels:
-      app: diags
-  template:
-    metadata:
-      labels:
-        app: diags
-    spec:
-      containers:
-      - name: diags
-        image: git.coopgo.io/coopgo-platform/diags:dev
-        ports:
-        - name: grpc
-          containerPort: 8080
-          protocol: TCP
-        volumeMounts:
-        - name: diags-config-volume 
-          mountPath: /config.yaml
-          subPath: config.yaml
-      volumes:
-      - name: diags-config-volume 
-        configMap:
-          name: diags-config
-          items:
-            - key: config.yaml
-              path: config.yaml

k8s_config/dev/diags/kustomization.yaml

@@ -1,8 +0,0 @@
-resources:
-- deployment.yaml 
-- service.yaml 
-
-configMapGenerator:
-- name: diags-config
-  files:
-    - config.yaml

k8s_config/dev/diags/service.yaml

@@ -1,11 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: diags
-spec:
-  selector:
-    app: diags
-  ports:
-  - port: 8080
-    name: grpc
-    targetPort: 8080

k8s_config/dev/kustomization.yaml

@@ -6,7 +6,6 @@ resources:
 - ./fleets/
 - ./groups-management/
 - ./mobility-accounts/
-- ./diags/
 
 # Apps
 - ./parcoursmob/

k8s_config/dev/mongodb/create_db.js

@@ -24,7 +24,7 @@ db.users.insert({
         "first_name" : "Admin",
         "last_name" : "Parcoursmob",
         "phone_number" : "+33600000000",
-        "email" : "noreply@parcoursmob.fr",
+        "email" : "admin@parcoursmob.fr",
         "groups" : [
             "483280d0-db2d-4f06-b361-02e4be5012d2",
             "483280d0-db2d-4f06-b361-02e4be5012d2:admin",
@@ -54,4 +54,4 @@ db.groups.insert({
         },
         "name" : "COOPGO"
     }
-})
+})

k8s_config/dev/parcoursmob/config.yaml

@@ -30,8 +30,6 @@ services:
       dial: fleets:8080
     agenda:
       dial: agenda:8080
-    diags:
-      dial: diags:8080
   navitia:
     api_key: abcd
 
@@ -99,7 +97,4 @@ modules:
       - name: Auto-école sociale (classique)
       - name: Information collective
       - name: Autre
-    documents_types:
-      - notice
-      - pieces
-      - other
+

k8s_config/dev/parcoursmob/deployment.yaml

@@ -13,7 +13,7 @@ spec:
     spec:
       containers:
       - name: parcoursmob
-        image: git.coopgo.io/coopgo-apps/parcoursmob:dev
+        image: git.coopgo.io/coopgo-apps/parcoursmob
         ports:
         - containerPort: 80
         volumeMounts:

k8s_config/k3d/kustomization.yaml

@@ -0,0 +1,12 @@
+resources: 
+  - ../dev/ 
+
+images:
+  - name: git.coopgo.io/coopgo-apps/parcoursmob
+    newName: "coopgo-apps/parcoursmob"
+  - name: git.coopgo.io/coopgo-platform/mobility-accounts
+    newName: "coopgo-platform/mobility-accounts"
+  - name: git.coopgo.io/coopgo-platform/fleets
+    newName: "coopgo-platform/fleets"
+  - name: git.coopgo.io/coopgo-platform/agenda
+    newName: "coopgo-platform/agenda"