User Tools

Site Tools


public:it:containers

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
public:it:containers [2022/04/01 11:55] – [Map local uid/gid into container] philpublic:it:containers [Unknown date] (current) – removed - external edit (Unknown date) 127.0.0.1
Line 1: Line 1:
-  * [[https://www.reddit.com/r/selfhosted/comments/s642av/deploy_compose_file_from_github/| Deploy compose file from github]] 
  
-  * [[https://github.com/Clivern/Peanut| Peanut: Peanut provides a REST API, Admin Dashboard and a command line tool to deploy and configure the commonly used services like databases, message brokers, graphing, tracing, caching tools ... etc. It perfectly suited for development, manual testing, automated testing pipelines where mocking is not possible and test drives.| 
- 
-  * [[https://www.nomadproject.io/docs/install|Nomad - Simple Kubernetes Alternative]] 
- 
-  * [[https://www.youtube.com/playlist?list=PLlVtbbG169nFr8RzQ4GIxUEznpNR53ERq|YT Playlist Titled Devops Master Class]] 
- 
-  * [[https://github.com/omaralsoudanii/jenkins-docker-ci|jenkins docker ci example]] 
- 
- 
-  * [[https://rootlesscontaine.rs/getting-started/common/subuid/| rootlesscontaine.rs - subuid ]] 
- 
-==== Exams ==== 
- 
-  * [[https://killer.sh/|Kubernetes CKS/CKA/CKAD Test simulator]] 
- 
- 
-==== Videos ==== 
-  * [[https://www.youtube.com/results?search_query=docker+swarm| Youtube search docker swarm]] 
-  * [[https://www.youtube.com/watch?v=pAM2GBCDGTo|Self-Hosting Your Homelab Services with SSL -- Let's Encrypt, MetalLB, Traefik, Rancher, Kubernetes]] 
- 
- 
-==== Nvidia-Docker ==== 
-  * https://github.com/keylase/nvidia-patch 
-  * https://emby.media/community/index.php?/topic/75330-best-gpu-for-transcoding/ 
-  * https://emby.media/community/index.php?/topic/76937-docker-hwa-nvidia-instructions/ 
- 
-==== Traefik ==== 
-  * [[https://www.youtube.com/watch?v=Gk9WER6DunE|Traefik Proxy v2.0 Docker Basic Tutorial]] 
-  * [[https://www.youtube.com/watch?v=C6IL8tjwC5E|Traefik Crash Course - Architecture, L7 & L4 Proxying, Weighted Round Robin, Enabling TLS 1.2/1.3]] 
-  * [[https://doc.traefik.io/traefik/routing/entrypoints/| Traefik Entrypoints]] 
-  * [[https://github.com/containous/traefik-library-image|Main github repository]] 
-  * [[https://github.com/IronicBadger/infra/blob/master/roles/ktz-traefik/templates/traefik.yaml.j2|traefik docker template example]] 
- 
-  * [[https://hollo.me/devops/routing-to-multiple-docker-compose-development-setups-with-traefik.html|Routing to multiple docker-compose development setups]] 
- 
-=== example config === 
-<code> 
---- 
-# step 0: 
-#   docker network create --driver=bridge --attachable  --internal=false gateway 
- 
-version: "3" 
- 
-services: 
-  traefik: 
-    image: traefik:v2.5.3 
-    container_name: traefik 
-    restart: unless-stopped 
-    ports: 
-      - "80:80" 
-      - "443:443" 
-      - "8080:8080" 
-      - "2222:2222" 
-    volumes: 
-      # Here is the mount of the local `ssl` directory 
-      - ./data/letsencrypt:/letsencrypt 
-      # The docker socket is mounted for auto-discovery of new services 
-      - /var/run/docker.sock:/var/run/docker.sock:ro 
-      # Globally Available User Creds 
-      - ./data/users_credentials:/users_credentials:ro 
-      # Mount config files directly 
-      - ./data/traefik-tls.toml:/traefik-tls.toml 
-      # traefik log file 
-      - /var/log/traefik:/var/log 
-    networks: 
-      - gateway 
-    command: 
-      - "--log.level=WARN" 
-      - "--accesslog=true" 
-      - "--accessLog.filePath=/var/log/access.log" 
-      #- "--accessLog.filters.statusCodes=400-499" 
-      #- "--pilot.dashboard=false" # Still being developed  
-      - "--api.dashboard=true" 
-      - "--api.insecure=true" 
-      - "--providers.docker=true" 
-      - "--providers.docker.exposedbydefault=false" 
-      - "--providers.docker.useBindPortIP=true" 
-      - "--providers.docker.network=gateway" 
-      - "--providers.docker.watch=true" 
-      # provide a file where options can be provided 
-      # Default TLS certificates must be specified in a file :sadface: 
-      - "--providers.file.filename=/traefik-tls.toml" 
-      # Entry Points 
-      - "--entrypoints.web.address=0.0.0.0:80" 
-      - "--entrypoints.websecure.address=0.0.0.0:443" 
-      - "--entrypoints.giteassh.address=0.0.0.0:2222" 
-      # import tls options from file for the websecure entrypoint 
-      - "--entrypoints.websecure.http.tls.options=default@file" 
-      # Certificate Resolver 
-      - "--certificatesresolvers.le.acme.email=user@example.com" 
-      - "--certificatesresolvers.le.acme.storage=/letsencrypt/acme.json" 
-      - "--certificatesresolvers.le.acme.caserver=https://acme-v02.api.letsencrypt.org/directory" 
-      # Staging. You'll want to replace the acme.json file. BACKUP this file!!! 
-      #- "--certificatesresolvers.le.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory" 
-      - "--certificatesresolvers.le.acme.httpchallenge=true" 
-      - "--certificatesresolvers.le.acme.httpchallenge.entrypoint=web" 
-    labels: 
-      - traefik.enable=true 
-      # Priority: Highest number == first prio 
-      #   https://doc.traefik.io/traefik/routing/routers/#priority 
-      # Route traffic for *.my.lan first 
-      - traefik.http.routers.mylan.rule=hostregexp(`.*\.my\.lan`) 
-      - traefik.http.routers.mylan.entrypoints=web 
-      - traefik.http.routers.mylan.priority=1000 
-      # global redirect to https 
-      - traefik.http.routers.http-catchall.rule=hostregexp(`{host:.+}`) 
-      - traefik.http.routers.http-catchall.entrypoints=web 
-      - traefik.http.routers.http-catchall.middlewares=redirect-to-https 
-      - traefik.http.routers.http-catchall.priority=1 
-      # middleware redirect 
-      - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https 
- 
-networks: 
-  gateway: 
-    external: true 
-</code> 
- 
-/traefik-tls.toml 
-<code> 
-# due to Go limitations, it is highly recommended that you use an ECDSA 
-# certificate, or you may experience compatibility issues 
-#[[tls.certificates]] 
-##certFile = "/path/to/signed_cert_plus_intermediates" 
-##keyFile = "/path/to/private_key" 
-#  certFile = "/traefik/certs/default.crt" 
-#  keyFile = "/traefik/certs/default.crt" 
- 
-[tls.options] 
-  [tls.options.default] 
-    minVersion = "VersionTLS12" 
-    cipherSuites = [ 
-      "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256", 
-      "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256", 
-      "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384", 
-      "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384", 
-      "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305", 
-      "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305" 
-    ] 
- 
-</code> 
-==== Min.io ==== 
-Minimal object store server with S3 compatibility written in GO. 
- 
-  * [[https://min.io/download#/linux | Main site and download]] 
-  * [[https://hub.docker.com/r/minio/minio/dockerfile|Docker Hub]] 
- 
-==== Garage ==== 
- 
-S3 compatible object store. Similar to min.io. 
- 
-  * [[https://git.deuxfleurs.fr/Deuxfleurs/garage|Git Repo]] 
-  * [[https://garagehq.deuxfleurs.fr/blog/2022-introducing-garage/ | Introducing Garage]] 
-  * [[https://garagehq.deuxfleurs.fr/documentation/design/benchmarks/|Benchmarks]] 
- 
-==== Kubernetes ==== 
- 
-  * [[https://www.reddit.com/r/devops/comments/pdqigh/how_to_write_kubernetes_yamls/|How to write Kubernetes yaml]] 
-  * [[https://blog.alexellis.io/bare-metal-kubernetes-with-k3s/|Bare-metal Kubernetes with K3S]] 
-  * [[https://rancher.com/docs/k3s/latest/en/cluster-access/|Rancher K3S Setup]] 
- 
-1. Run this on hypervisor: 
-<code>curl -sfL https://get.k3s.io | sh -</code> 
- 
-2. Verify it worked: 
-<code>k3s kubectl get node</code> 
- 
-Docs: https://rancher.com/docs/k3s/latest/en/cluster-access/ 
- 
- 
-3. Copy ''%%/etc/rancher/k3s/k3s.yaml%%'' on the hypervisor to your machine at ''%%~/.kube/config%%''. Then replace “localhost” with the IP or name of your K3s server. ''%%kubectl%%'' can now manage your K3s cluster. 
-===== Docker ===== 
-  * [[https://www.youtube.com/watch?v=5cNrTU6o3Fw|Pods and Containers - Kubernetes Networking | Container Communication inside the Pod]] 
-  * [[https://www.splitgraph.com/blog/docker-compose-production|Blog post - Docker Compose in Production]] 
-  * [[https://developers.redhat.com/blog/2020/11/19/transitioning-from-docker-to-podman/| Docker to Podman]] 
- 
- 
-==== Deploy docker containers on remote hosts with docker-compose ==== 
-  * [[https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/|Deploy docker containers on a remote host with docker-compose]] 
- 
- 
-=== Environment variable === 
- 
-    DOCKER_HOST=“ssh://user@remotehost” docker-compose up -d 
- 
-=== Context === 
- 
-<code> 
-$ docker context ls 
-NAME   DESCRIPTION   DOCKER ENDPOINT   KUBERNETES ENDPOINT   ORCHESTRATOR 
- 
-remote               ssh://user@remotemachine 
-$ cd hello-docker 
-$ docker-compose ‐‐context remote up -d 
-</code> 
- 
-<code> 
-$ docker context create remote ‐‐docker “host=ssh://user@remotemachine” 
-remote 
-Successfully created context “remote” 
- 
-$ docker context ls 
-NAME      DESCRIPTION            DOCKER ENDPOINT    KUBERNETES ENDPOINT     ORCHESTRATOR 
-default * Current DOCKER_HOST…   unix:///var/run/docker.sock                swarm 
-remote                           ssh://user@remotemachine 
-</code> 
- 
- 
- 
- 
-=== Format commands === 
-[[https://container42.com/2016/03/27/docker-quicktip-7-psformat/|source]] 
- 
-Paste this into ''%%~/.docker/config.json%%'': 
-<code> 
-{ 
-  "psFormat": "table {{.Names}}\\t{{.Image}}\\t{{.RunningFor}} ago\\t{{.Status}}\\t{{.Command}}", 
-  "imagesFormat": "table {{.Repository}}\\t{{.Tag}}\\t{{.ID}}\\t{{.Size}}" 
-} 
-</code> 
- 
- 
-===== Podman ===== 
-Seems to not require mapping UID and GIDs. 
- 
-Ubuntu 20.10 supports podman install. 
- 
-If on 20.04 you can add the repo: https://podman.io/getting-started/installation 
- 
-[[https://developers.redhat.com/blog/2020/09/25/rootless-containers-with-podman-the-basics/|Rootless Podman basics]] 
- 
-  - On ubuntu 20.04 and earlier install the podman repo. 
-  - Install<code> 
-apt-get install -y slirp4netns podman 
-</code> 
-  - Edit max user name spaces <code> 
-$ echo “user.max_user_namespaces=28633” > /etc/sysctl.d/userns.conf   
-$ sysctl -p /etc/sysctl.d/userns.conf 
-</code> 
- 
-[[https://podman.io/getting-started/installation#configuration-files|Podman Config files]] 
-[[https://developers.redhat.com/blog/2020/11/19/transitioning-from-docker-to-podman/| Podman to Docker]] 
- 
- 
-==== subuid and subgid ==== 
- 
-  * https://rootlesscontaine.rs/getting-started/common/subuid/ 
- 
-Pre-generating all possible values for /etc/subuid and /etc/subgid, based on uid and gid, rather than the user and group names, is also possible. This can simplify shared management of shared computing environments using LDAP/AD, while there is no standardized way to store or retrieve subuid and subgid values from those directories. 
- 
-An example python program to generate the files: 
- 
-<code> 
-f = open("/etc/subuid", "w") 
-for uid in range(1000, 65536): 
-    f.write("%d:%d:65536\n" %(uid,uid*65536)) 
-f.close() 
- 
-f = open("/etc/subgid", "w") 
-for uid in range(1000, 65536): 
-    f.write("%d:%d:65536\n" %(uid,uid*65536)) 
-f.close() 
-</code> 
- 
- 
-==== Map local uid/gid into container ====  
- 
-  * https://github.com/containers/podman/blob/main/troubleshooting.md#33-container-creates-a-file-that-is-not-owned-by-the-users-regular-uid 
- 
-<code> 
-#!/bin/bash 
- 
-# https://github.com/containers/podman/blob/main/troubleshooting.md#33-container-creates-a-file-that-is-not-owned-by-the-users-regular-uid 
- 
-subuidSize=$(( $(podman info --format "{{ range .Host.IDMappings.UIDMap }}+{{.Size }}{{end }}" ) - 1 )) 
-subgidSize=$(( $(podman info --format "{{ range .Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 )) 
- 
-uid=$(id -u) 
-gid=$(id -g) 
- 
-podman run --rm -it \ 
-  -v ./dir1:/dir1 \ 
-  --user $uid:$gid \ 
-  --uidmap $uid:0:1 \ 
-  --uidmap 0:1:$uid \ 
-  --uidmap $(($uid+1)):$(($uid+1)):$(($subuidSize-$uid)) \ 
-  --gidmap $gid:0:1 \ 
-  --gidmap 0:1:$gid \ 
-  --gidmap $(($gid+1)):$(($gid+1)):$(($subgidSize-$gid)) \ 
-  docker.io/library/ubuntu /bin/bash -c "touch /dir1/a; chmod 600 /dir1/a" 
-</code> 
- 
-=== Docker features that are unsupported in Podman === 
- 
- 
- 
- 
-[[https://github.com/containers/podman-compose|podman-compose]] 
- 
- 
- 
- 
-[[https://github.com/j8r/dockerfiles/tree/master/systemd/ubuntu| Ubuntu Container with systemd]] 
- 
- 
- 
- 
- 
-===== app container ===== 
-https://github.com/google/gvisor 
-https://firecracker-microvm.github.io/ 
- 
- 
-===== DevOps discussion ===== 
-https://www.reddit.com/r/devops/comments/p11l4k/devops_engineer_is_the_new_sysadmin/ 
-https://medium.com/nerd-for-tech/devops-engineer-is-the-new-sysadmin-5bc46b86d413 
public/it/containers.1648832116.txt.gz · Last modified: 2022/04/01 11:55 by phil