ci/fastboot: Add a serial timeout to catch fastboot prompt failure.

The a530s will occasionally fail to make it to the fastboot prompt,
with no other deltas between a working log and a log stalled waiting
for that line to show up.

So, add a serial timeout (like the rpi boards do for similar reasons),
and on timeout restart the run.  We actually restart the whole serial
watching process, because the SerialBuffer finishes itself on timeout.
This should also help with the intermittent issue we've had where a
power cycle causes the python serial module to throw an exception.

Tested with the gitlab-disabled db820c that never makes it to the
fastboot prompt (I think it's one where we need a longer micro cable
to connect it!) and saw successful boot looping to retry.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11308>

1 files changed, 8 insertions, 4 deletions

diff --git a/.gitlab-ci/bare-metal/fastboot_run.py b/.gitlab-ci/bare-metal/fastboot_run.py
index eb0b7d4e8ad..5def91aa183 100755
--- a/.gitlab-ci/bare-metal/fastboot_run.py
+++ b/.gitlab-ci/bare-metal/fastboot_run.py
@@ -31,7 +31,9 @@ import threading
 class FastbootRun:
     def __init__(self, args):
         self.powerup = args.powerup
-        self.ser = SerialBuffer(args.dev, "results/serial-output.txt", "R SERIAL> ")
+        # We would like something like a 1 minute timeout, but the piglit traces
+        # jobs stall out for long periods of time.
+        self.ser = SerialBuffer(args.dev, "results/serial-output.txt", "R SERIAL> ", timeout=600)
         self.fastboot="fastboot boot -s {ser} artifacts/fastboot.img".format(ser=args.fbserial)
 
     def print_error(self, message):
@@ -59,7 +61,7 @@ class FastbootRun:
 
         if not fastboot_ready:
             self.print_error("Failed to get to fastboot prompt")
-            return 1
+            return 2
 
         if self.logged_system(self.fastboot) != 0:
             return 1
@@ -87,8 +89,8 @@ class FastbootRun:
                 else:
                     return 1
 
-        self.print_error("Reached the end of the CPU serial log without finding a result")
-        return 1
+        self.print_error("Reached the end of the CPU serial log without finding a result, restarting run...")
+        return 2
 
 def main():
     parser = argparse.ArgumentParser()
@@ -105,6 +107,8 @@ def main():
         if retval != 2:
             break
 
+        fastboot = FastbootRun(args)
+
     fastboot.logged_system(args.powerdown)
 
     sys.exit(retval)