SolusVM 2 unavailable: postgresql container fails to start: PANIC could not locate a valid checkpoint record

Have more questions? Submit a request

Applicable to:

  • SolusVM 2

Symptoms 

  • SolusVM 2 UI is unavailable
  • Postgresql container in the docker stack fails to start. The following error is displayed in the service log:

    # docker service logs solus_postgresql
    ...
    PANIC: could not locate a valid checkpoint record 

Cause

PostgreSQL database is corrupted.

Resolution

  1. Connect to SolusVM 2 management node via SSH
  2. Backup SolusVM 2
  3. Stop solus stack:

    # docker stack rm solus

  4. Find the postgresql image ID:

    # docker image ls | grep postgres | awk {'print $3'}
    d029edc38682

  5. Started a temporary container with the image ID from step 3:

    # docker run -it --rm --entrypoint /bin/bash -v /usr/local/solus/postgresql/12/data:/var/lib/postgresql/data <IMAGE ID>

    where <IMAGE ID> is the corresponding postgresql image ID from step 3. For example:

    # docker run -it --rm --entrypoint /bin/bash -v /usr/local/solus/postgresql/12/data:/var/lib/postgresql/data d029edc38682

  6. Impersonate as the user 'postgres':

    # su postgres

  7. Regenerate the entrypoint with the command:

    # pg_resetwal -f /var/lib/postgresql/data

  8. Exit the container by pressing 'CTRL +D' on keyboard twice.
  9. Restart docker and start 'solus' stack:

    # service docker restart && docker stack deploy --with-registry-auth -c /usr/local/solus/config/stack.yml solus

Articles in this section

Was this article helpful?
0 out of 0 found this helpful
Share

Comments

0 comments

Please sign in to leave a comment.