RT thread seems blocked
Bradley Valdenebro Peter (DC-AE/ESW52)
Peter.BradleyValdenebro at boschrexroth.nl
Tue Mar 24 12:43:17 CET 2020
We run some tests during the weekend and although we have less occurrences we still indeed see them.
One interesting thing we see besides the 1ms stalling is that sometimes we see over 100us between IRQ and ISR start.
At this point we do not know how to proceed further. Any help or suggestion is greatly appreciated.
Met vriendelijke groeten / Best regards,
Tel. +31 40 257-8858
From: Bradley Valdenebro Peter (DC-AE/ESW52)
Sent: 20 March 2020 11:14
To: 'Jan Kiszka' <jan.kiszka at siemens.com>; xenomai at xenomai.org
Subject: RE: RT thread seems blocked
Following your advice we changed our setup and we are now running the following:
$ uname -a
Linux wom03 4.14.110-mainline #1 SMP PREEMPT Tue Mar 17 17:25:22 CET 2020 armv7l GNU/Linux
$ cat /proc/xenomai/version
$ cat /proc/ipipe/version
It is still early to say but it is looking better.
We will keep you updated.
From: Jan Kiszka <jan.kiszka at siemens.com>
Sent: 17 March 2020 11:45
To: Bradley Valdenebro Peter (DC-AE/ESW52) <Peter.BradleyValdenebro at boschrexroth.nl>; xenomai at xenomai.org
Subject: Re: RT thread seems blocked
On 17.03.20 11:22, Bradley Valdenebro Peter (DC-AE/ESW52) via Xenomai wrote:
> Hello Xenomai team,
> We are running a Xenomai/Linux setup on a Zynq Z-7020 SoC (We run Linux on CPU0 and Xenomai on CPU1):
> - Linux version 4.4.0-xilinx (gcc version 8.3.0 (Buildroot 2019.02-00080-gc31d48e) ) #1 SMP PREEMPT
> - ipipe ARM patch #8
> - Xenomai 3.0.10
> Lately we have been experiencing that our highest priority real time Xenomai thread halts for around 1ms every now and then. We see it happening during different points in our thread.
> This thread is triggered by an interrupt generated from the Zynq fpga logic every 125us. We have an fpga counter which is incremented every time an interrupt is generated.
> We can see in our logging that we can miss up to 4 counts (~1ms) and we do not know what is going on during that time.
> We have tried using the ipipe-tracer but that gives some overhead and the issue isn't reproducible.
> The same with ftrace.
> Switching from Xenomai 3.0.10 to 3.1 we see the number of occurrences decrease but still we see them.
> It is quite difficult to reproduce. It happens around once every 24 hours.
> Have you got any ideas?
> Are there any known Xenomai/ipipe bugs that could explain this behaviour?
Already tried ipipe-4.4.y-cip head? I can't promise it helps, but it does contain quite a few fixes, compared to your baseline.
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
More information about the Xenomai