Changing gitlab-runner configuration options

It is possible to change the configuration of a running gitlab-runner by editing the file ~/.gitlab-runner/config.toml.

For example to be able to switch the log_level from ‘info’ to ‘debug’ and back again, you can do so by logging into the gitlab-runner and then editing the file.

The file is reloaded without the need for rebooting the gitlab-runner.

Google Autopilot and Gitlab failed builds

Problem

You want to use Google’s Autopilot for your gitlab runners, but your job/builds fail because of low resources (ie ephemeral storage).

Solution

You can use a limit range to increase the limits for ephemeral storage or/and memory that will make Google’s autopilot to use them and scale them appropriately.

Create a limit range file like:

apiVersion: v1
kind: LimitRange
metadata:
  name: limit-ephemeral-storage
spec:
  limits:
  - default:
      ephemeral-storage: "10Gi"
      memory: "16Gi"
    defaultRequest:
      ephemeral-storage: "10Gi"
      memory: "16Gi"
    type: Container

And then apply it to your cluster

kubectl -n namespace apply -f limit_range.yaml

googlecloudsdk.calliope.exceptions.HttpException: ResponseError: code=400, message=Autopilot clusters must be regional clusters.

Problem

Trying to create an auto-cluster either using terraform or gcloud cli, and specifying the region name returns the error that ‘Autopilot clusters must be regional clusters.

So with gcloud this is the command and output

kosmas: (master %)$ gcloud container clusters create-auto test-cluster --region=europe-west6-b
Note: The Pod address range limits the maximum size of the cluster. Please refer to https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr to learn how to optimize IP address allocation.
ERROR: (gcloud.container.clusters.create-auto) ResponseError: code=400, message=Autopilot clusters must be regional clusters.

Solution

Using the actual region name (that can be taken from the list of available zones/regions)

gcloud compute zones list

NAME                       REGION                   STATUS  NEXT_MAINTENANCE  TURNDOWN_DATE
us-east1-b                 us-east1                 UP
us-east1-c                 us-east1                 UP
...
europe-west6-b             europe-west6             UP
...

And using the correct region name (without the b)

gcloud container clusters create-auto test-cluster --region=europe-west6 --verbosity debug

...
Created [https://container.googleapis.com/v1/projects/gitlab-runner-343714/zones/europe-west6/clusters/test-cluster].
...
NAME          LOCATION      MASTER_VERSION   MASTER_IP     MACHINE_TYPE  NODE_VERSION     NUM_NODES  STATUS
test-cluster  europe-west6  1.21.6-gke.1503  xxx.xxx.xxx.xxx  e2-medium     1.21.6-gke.1503  3          RUNNING

Adding a LOKI report in Grafana counting number of http_user_agents from NGINX logs

To add a Loki report in Grafana, as for example having a count of the different http_user_agents that have logged in, you can use the following query:

count by (http_agent) (rate({namespace="ingress-nginx",stream="stdout"} |= "https://domain.name.com/session/new" |~ "GET\\s/\\s" | pattern "<ip> - - [<timestamp>] \"<method> <path> <version>\" <result> <_> \"<url>\" \"<http_agent>\" <_>" [$__interval]))

You select the source (Loki) and then use the {namespace=”ingress-nginx”,stream=”stdout”} to get the log files from Nginx.

Then you can filter by two conditions:

  1. the instance login
|= "https://domain.name.com/session/new"

and

  1. use the regular expression to only select the GET / calls
|~ "GET\\s/\\s"

Then you can use the pattern parser for the log file in order to get the label

pattern "<ip> - - [<timestamp>] \"<method> <path> <version>\" <result> <_> \"<url>\" \"<http_agent>\" <_>"

<_> is used if we are not interested in getting the field, so everything after http_agent is not analyzed in this case.

Finally use the

count by(http_agent) 

to get the data (note that you also need to use the rate with the [$__interval])

Convert a single line string ‘a b c’ to a multiple line string with double quotes and commas like [“a”, “b”, “c”] using bash and sed

Problem

You have single line output that contains multiple strings (could be output from a kubectl that gives all the instance names) and you want to convert it to text that can be used as the a variable enclosed by double quotes, one string on one line and separated by commas.

So having the string in a file called input.txt:

a b c

you want to convert it to the following and save it in another file called output.txt

"a",
"b",
"c"


Solution

You can use the following:

cat input.txt | tr ' ' '\n' | sed 's/^/"/g' | sed 's/$/",/g' > output.txt

which will first replace the spaces separating the strings to newlines, and then use two passes with sed, in the first adding the first quote, and the second adding the second quote and the comma.