Dwellers: Difference between revisions

From Nasqueron Agora
No edit summary
 
(40 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''Dwellers''' is an VMWare EXSi instance installed on [[Stormshear]].
'''Dwellers''' is an VMWare ESXi instance installed on [[hyper-001]].


The goal of this server is to provide a Docker / OpenShift / Geard CentOS PaaS service.
The goal of this server is to provide a development environment for Docker images and deployments.
 
The historical goal of this server were to provide a Docker / OpenShift / Geard CentOS PaaS service, and host LXC.


== Basic information ==
== Basic information ==
* '''IPs:'''
* '''IPs:'''
** 212.129.32.223
** 51.255.124.11 (SSH doesn't listen to public IPv4)
** 2001:470:1f13:ce7:ca5:cade:fab:1e
** 2001:470:1f13:30b:ca5:cade:fab:1e
** 172.27.27.4 (use this for SSH)
* '''Hostname:''' dwellers.nasqueron.org
* '''Hostname:''' dwellers.nasqueron.org
* '''Homepage:''' http://dwellers.nasqueron.org/
* '''Homepage:''' https://dwellers.nasqueron.org/
* '''Configuration:'''Access to 3.5 GB RAM and 4 core, burstable on request to 8 cores/+-6 Gb (to be negotiated according [[Ysul]] use)
* '''Configuration:''' 8 GB RAM and 8 core
* '''OS:''' CentOS 7
* '''OS:''' CentOS Stream
* '''ISP:''' [http://www.online.net Online] (FR)
* '''ISP:''' OVH (FR)
* '''Network:''' Illiad (FR)
* '''Status:''' Used for Docker development and testing
* '''Status:''' Installing.
* '''Policy:''' Access for any Nasqueron or Wolfplex project
* '''Policy:''' Access for any Nasqueron or Wolfplex project
* '''Started:''' 2014-07-13
* '''Started:''' 2014-07-13


== Services ==
== Services ==
* SSH (*:22)
* SSH (172.27.27.4:22)
* Docker
* Docker
* OpenShift
** [[Operations grimoire/Mastodon]]
** RabbitMQ, Nasqueron Notifications intelligent bus
* <strike>OpenShift</strike>
* <strike>LXC</strike>


== Containers ==
== Containers ==
=== Ports table ===
=== Ports table ===
Ø indicates an unmapped port. In such cases, it's accessible logging in Dwellers, and connecting locally to the current mutable container IP variable and the immutable specified port.


{| class="wikitable"
The ports table is composed of the ''reserved-for-legacy-docker-migration-'' entries in {{Ops file|PORTS}}.
! '''Container name'''
! '''Container image'''
! '''Prefix'''
! '''Service'''
! '''Internal port'''
! '''External port'''
|-
| Dwellers Shipyard
| shipyard/shipyard
| 30
| Apache||80||30080
|-
|rowspan=3| phabricator.nasqueron.org
|rowspan=3| yesnault/docker-phabricator:latest
|rowspan=3| 31
| SSH||22||Ø
|-
| Apache||80||31080
|-
| MySQL||3306||Ø
|-
|rowspan=3| forum.nasqueron.org
|rowspan=3| provided by the project Discuss docker image
|rowspan=3| 32
| SSH||22||Ø
|-
| Apache||80||32080
|-
|colspan=3|Others services like redis are still to document
|-
|rowspan=2| bugzilla.espace-win.org
|rowspan=2| dklawren/docker-bugzilla
|rowspan=2| 33
| SSH||22||Ø
|-
| Apache||80||33080
|-


|}
=== forum.nasqueron.org ===
'''Port prefix:''' 32


=== phabricator.nasqueron.org ===
Provides a Discourse instance, to be used as a forum at http://forum.nasqueron.org/
'''Port prefix:''' 31


Provides a Phabricator instance for Nasqueron projects at [http://phabricator.nasqueron.org http://phabricator.nasqueron.org].
First, update the configuration:
    cd /data/discourse/app/
    #ensure you have id_zr in the SSH agent with ssh-add -l
    #if not, and if you have trouble with an agent, alias ssh "ssh -i /root/.ssh/id_zr" should work
    make update


To run a new container:
Then, launch db and cache containers:
    docker run -p 31080:80 nasqueron-phabricator
    docker run -d -v /data/discourse/postgres:/var/lib/postgresql -e LC_ALL=C.UTF-8 --name=discourse-postgres postgres
    docker run -d --name discourse-redis redis


=== Shipyard ===
Finally, launch web container:
'''Port prefix:''' 30
    docker run -d -v /data/discourse/app:/data/config -p 32000:3000 -p 32080:80 --link discourse-postgres:db --link discourse-redis:cache --name discourse-web nasqueron/discourse


Run RethinkDB for the storage and launch shipyard:
At launch time, the web container can perform some tasks. Erase the dotfiles in /data/discourse/app/ to force them:
    docker run -it -d --name shipyard-rethinkdb-data --entrypoint /bin/bash shipyard/rethinkdb -l
* rake db:migrate if .database-initialized is not found
    docker run -it -P -d --name shipyard-rethinkdb --volumes-from shipyard-rethinkdb-data shipyard/rethinkdb
* rake assets:precompile if .database-initialized is not found
    docker run -it -p 30080:8080 -d --name shipyard --link shipyard-rethinkdb:rethinkdb shipyard/shipyard
* regenerate language configuration and files if .language-set not found and a language file contains a language string


To control shipyard instance, launch the CLI (also in a container):
So to switch from English to French for example:
     docker run -it shipyard/shipyard-cli
     echo fr > language
    rm .language-set


Documentation is at http://shipyard-project.com/docs/usage/cli/
Also at launch time, the web container will populate config directory with missing config files before to create symlinks of these files to the Discourse web config folder. So, if you found a Discourse instance trying to find a database at localhost, don't forget to generate from discourse.conf.tmpl a discourse.conf file following instructions given above in update the configuration step.


== Troubleshoot ==
== Troubleshoot ==
=== How to point a domain here? ===
=== How to point a domain here? ===
For your domains:
For your domains:
* subdomain.domain.tld A 212.129.32.223
* subdomain.domain.tld A 51.255.124.11
* subdomain.domain.tld AAAA 2001:470:1f13:ce7:ca5:cade:fab:1e
* subdomain.domain.tld AAAA 2001:470:1f13:30b:ca5:cade:fab:1e


To request a DNS update for domains using extensively the Nasqueron servers infrastructure:
To request a DNS update for domains using extensively the Nasqueron servers infrastructure:
Line 110: Line 83:
     docker inspect <instance id> #gets the local IP
     docker inspect <instance id> #gets the local IP
     ssh <IP 172.*>
     ssh <IP 172.*>
If you don't see the IP with docker inspect, check you use the instance id, not the image name.
'''Note: with recent Docker versions, you don't need to SSH anymore: you can use docker exec -it <container name> <your favorite shell> instead.'''


=== No network at boot time ===
=== No network at boot time ===
Line 119: Line 96:


;If you've reset the configuration and need to add again the IP:
;If you've reset the configuration and need to add again the IP:
     ip addr 212.129.32.223/32 dev ens192
     ip addr add 51.255.124.11/32 dev ens192


;Routing is probably the issue:
;Routing is probably the issue:
     ip route add 62.210.76.1 dev ens192
     ip route add 91.121.86.254 dev ens192
     ip route add default via 62.210.76.1
     ip route add default via 91.121.86.254


;Same for the case we can ping/ssh (slowly) from [[Ysul]] but not from the world:
;Same for the case we can ping/ssh (slowly) from [[Ysul]] but not from the world:
     ip route change 62.210.76.1 dev ens192
     ip route change 91.121.86.254 dev ens192
     ip route change default via 62.210.76.1
     ip route change default via 91.121.86.254


;Reconfigure the IPv6 tunnel
;Reconfigure the IPv6 tunnel
At some point, the Linux route2 method stopped to work, but the Linux net-tools method still work.
     ip tunnel del he-ipv6
     ip tunnel del he-ipv6
     ip tunnel add he-ipv6 mode sit remote 216.66.84.42 local 212.129.32.223 ttl 255
     ifconfig sit0 up
     ip link set he-ipv6 up
    ifconfig sit0 inet6 tunnel ::216.66.84.42
     ip addr add 2001:470:1f12:ce7::2/64 dev he-ipv6
     ifconfig sit1 up
     ip addr add 2001:470:1f13:ce7:ca5:cade:fab:1e/64 dev he-ipv6
     ifconfig sit1 inet6 add 2001:470:1f12:30b::2/64
     ip route change ::/0 dev he-ipv6
     ifconfig sit1 inet6 add 2001:470:1f13:30b:ca5:cade:fab:1e/64
     route -A inet6 add ::/0 dev sit1
 
Issue reproducible on a fresh CentOS 8.1 installation.


=== A port on the host doesn't reply (but does in Docker) ===
=== A port on the host doesn't reply (but does in Docker) ===
You can reset the iptables configuration. A script has been provided for that.
You can reset the iptables configuration. A script has been provided for that.


     # systemctl stop docker
     $ systemctl stop docker
     # [[reset-iptables]]
     $ /usr/sbin/[[reset-iptables]]
     # systemctl start docker
     $ systemctl start docker
 
If you're willing to restrict ports, you can use instead /root/reset-iptables-dwellers
 
You then need to reapply the mailserver network iptables rules.
 
=== From Docker and LXC, it's not possible to connect outside ===
 
It could be the net.ipv4.ip_forward switched from 1 to 0:
 
    sysctl net.ipv4.ip_forward=1
 
Or it could be an issue with iptables:
 
    systemctl stop docker
    iptables -t nat -F POSTROUTING
    iptables -t nat -A POSTROUTING -o ens192 -j MASQUERADE
    systemctl start docker
 
If you need to recreate the forwarding map ([http://devcentral.nasqueron.org/P91 P91]):
 
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 25 -j DNAT --to-destination 10.0.3.8:25
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 21080 -j DNAT --to-destination 10.0.3.8:80
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 110 -j DNAT --to-destination 10.0.3.8:110
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 143 -j DNAT --to-destination 10.0.3.8:143
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 465 -j DNAT --to-destination 10.0.3.8:465
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 587 -j DNAT --to-destination 10.0.3.8:587
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 993 -j DNAT --to-destination 10.0.3.8:993
    iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 995 -j DNAT --to-destination 10.0.3.8:995


[[Category:Dwellers]]
[[Category:Dwellers]]

Latest revision as of 00:22, 25 May 2023

Dwellers is an VMWare ESXi instance installed on hyper-001.

The goal of this server is to provide a development environment for Docker images and deployments.

The historical goal of this server were to provide a Docker / OpenShift / Geard CentOS PaaS service, and host LXC.

Basic information

  • IPs:
    • 51.255.124.11 (SSH doesn't listen to public IPv4)
    • 2001:470:1f13:30b:ca5:cade:fab:1e
    • 172.27.27.4 (use this for SSH)
  • Hostname: dwellers.nasqueron.org
  • Homepage: https://dwellers.nasqueron.org/
  • Configuration: 8 GB RAM and 8 core
  • OS: CentOS Stream
  • ISP: OVH (FR)
  • Status: Used for Docker development and testing
  • Policy: Access for any Nasqueron or Wolfplex project
  • Started: 2014-07-13

Services

Containers

Ports table

The ports table is composed of the reserved-for-legacy-docker-migration- entries in rOPS: PORTS.

forum.nasqueron.org

Port prefix: 32

Provides a Discourse instance, to be used as a forum at http://forum.nasqueron.org/

First, update the configuration:

    cd /data/discourse/app/
    #ensure you have id_zr in the SSH agent with ssh-add -l
    #if not, and if you have trouble with an agent, alias ssh "ssh -i /root/.ssh/id_zr" should work
    make update

Then, launch db and cache containers:

    docker run -d -v /data/discourse/postgres:/var/lib/postgresql -e LC_ALL=C.UTF-8 --name=discourse-postgres postgres
    docker run -d --name discourse-redis redis

Finally, launch web container:

    docker run -d -v /data/discourse/app:/data/config -p 32000:3000 -p 32080:80 --link discourse-postgres:db --link discourse-redis:cache --name discourse-web nasqueron/discourse

At launch time, the web container can perform some tasks. Erase the dotfiles in /data/discourse/app/ to force them:

  • rake db:migrate if .database-initialized is not found
  • rake assets:precompile if .database-initialized is not found
  • regenerate language configuration and files if .language-set not found and a language file contains a language string

So to switch from English to French for example:

    echo fr > language
    rm .language-set

Also at launch time, the web container will populate config directory with missing config files before to create symlinks of these files to the Discourse web config folder. So, if you found a Discourse instance trying to find a database at localhost, don't forget to generate from discourse.conf.tmpl a discourse.conf file following instructions given above in update the configuration step.

Troubleshoot

How to point a domain here?

For your domains:

  • subdomain.domain.tld A 51.255.124.11
  • subdomain.domain.tld AAAA 2001:470:1f13:30b:ca5:cade:fab:1e

To request a DNS update for domains using extensively the Nasqueron servers infrastructure:

  • subdomain.nasqueron.org CNAME www3.nasqueron.org
  • subdomain.espace-win.org CNAME www2.espace-win.org

How to access by SSH to an instance?

See the ports table to check if a port is assigned. We don't assign port if there is no reason general public got access to the VM by SSH. We assign port each time a stable address is needed (for example to talk with a Git server)

If the port is mapped:

   ssh -p <port> username@dwellers.nasqueron.org

If the port is unmapped, you can from Dwellers:

   docker ps
   docker inspect <instance id> #gets the local IP
   ssh <IP 172.*>

If you don't see the IP with docker inspect, check you use the instance id, not the image name.

Note: with recent Docker versions, you don't need to SSH anymore: you can use docker exec -it <container name> <your favorite shell> instead.

No network at boot time

Access the machine on the hypervisor, then:

Check the interface is up
   ip addr
   ifup ens192 # to bring it up
If you've reset the configuration and need to add again the IP
   ip addr add 51.255.124.11/32 dev ens192
Routing is probably the issue
   ip route add 91.121.86.254 dev ens192
   ip route add default via 91.121.86.254
Same for the case we can ping/ssh (slowly) from Ysul but not from the world
   ip route change 91.121.86.254 dev ens192
   ip route change default via 91.121.86.254
Reconfigure the IPv6 tunnel

At some point, the Linux route2 method stopped to work, but the Linux net-tools method still work.

   ip tunnel del he-ipv6
   ifconfig sit0 up
   ifconfig sit0 inet6 tunnel ::216.66.84.42
   ifconfig sit1 up
   ifconfig sit1 inet6 add 2001:470:1f12:30b::2/64
   ifconfig sit1 inet6 add 2001:470:1f13:30b:ca5:cade:fab:1e/64
   route -A inet6 add ::/0 dev sit1

Issue reproducible on a fresh CentOS 8.1 installation.

A port on the host doesn't reply (but does in Docker)

You can reset the iptables configuration. A script has been provided for that.

   $ systemctl stop docker
   $ /usr/sbin/reset-iptables
   $ systemctl start docker

If you're willing to restrict ports, you can use instead /root/reset-iptables-dwellers

You then need to reapply the mailserver network iptables rules.

From Docker and LXC, it's not possible to connect outside

It could be the net.ipv4.ip_forward switched from 1 to 0:

   sysctl net.ipv4.ip_forward=1

Or it could be an issue with iptables:

   systemctl stop docker
   iptables -t nat -F POSTROUTING
   iptables -t nat -A POSTROUTING -o ens192 -j MASQUERADE
   systemctl start docker

If you need to recreate the forwarding map (P91):

   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 25 -j DNAT --to-destination 10.0.3.8:25
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 21080 -j DNAT --to-destination 10.0.3.8:80
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 110 -j DNAT --to-destination 10.0.3.8:110
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 143 -j DNAT --to-destination 10.0.3.8:143
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 465 -j DNAT --to-destination 10.0.3.8:465
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 587 -j DNAT --to-destination 10.0.3.8:587
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 993 -j DNAT --to-destination 10.0.3.8:993
   iptables -t nat -I PREROUTING -i ens192 -p TCP -d 51.255.124.11/32 --dport 995 -j DNAT --to-destination 10.0.3.8:995