[Xenomai] i.MX6q memory write causes high latency

[Federico Sbalchiero]

Il 05/07/2018 11:14, Philippe Gerum ha scritto:
> On 07/04/2018 07:06 PM, Federico Sbalchiero wrote:
>> Hi,
>> first I want to say thanks to everyone involved in Xenomai for their job.
>>
>> I'm testing Xenomai 3.0.7 and ipipe-arm/4.14 on Freescale/NXP i.MX6q
>> sabresd board using Yocto. System boots fine and is stable, but latency
>> under load (xeno-test) is higher than in my reference system (Xenomai
>> 2.6.5 on Freescale kernel 3.10.17 + ipipe 3.10.18).
>> This is after disabling power management, frequency scaling, CMA,
>> graphics, tracing, debug.
>>
>> I have found that a simple non-realtime user space process writing a
>> buffer in memory (memwrite) is able to trigger such high latencies.
>> Latency worsen a lot running a copy of the process on each core.
>> There is a correlation between buffer size and cache size suggesting
>> an L2 cache issue, like the L2 write allocate discussed in the mailing
>> list, but I can confirm L2 WA is disabled (see log).
>>
>> I'm looking for comments or suggestions.
>>
> A basic dd if=/dev/zero of=/dev/null loop in the background is enough to
> raise the latency actually. Could you try the Xenomai 3 + 3.18 combo on
> your hw and let us know whether you see the same regression?
>
> TIA,
>

in the same configuration (kernel 3.18.20-ipipe + xenomai 3.0.7)
dd if=/dev/zero of=/dev/null has almost no effect on latency.
I think all data write to a few small buffers, not stressing L2
cache.

latency under load (four dd if=/dev/zero of=/dev/null instances)

RTT|  00:00:01  (periodic user-mode task, 1000 us period, priority 99)
RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat 
best|--lat worst
RTD|      1.093|      2.078|      7.757|       0| 0|      1.093|      7.757
RTD|      1.161|      2.090|      5.391|       0| 0|      1.093|      7.757
RTD|      1.128|      2.079|      6.537|       0| 0|      1.093|      7.757
RTD|      1.155|      2.140|      9.926|       0| 0|      1.093|      9.926
RTD|      1.117|      2.086|      9.483|       0| 0|      1.093|      9.926
RTD|      1.130|      2.079|      6.362|       0| 0|      1.093|      9.926
RTD|      1.165|      2.069|      5.506|       0| 0|      1.093|      9.926
RTD|      1.119|      2.066|      5.304|       0| 0|      1.093|      9.926
RTD|      1.202|      2.092|     10.538|       0| 0|      1.093|     10.538
RTD|      1.139|      2.071|      5.649|       0| 0|      1.093|     10.538
RTD|      1.169|      2.080|      5.604|       0| 0|      1.093|     10.538
RTD|      1.134|      2.065|      5.586|       0| 0|      1.093|     10.538
RTD|      1.027|      2.073|      5.452|       0| 0|      1.027|     10.538
RTD|      1.103|      2.188|      9.456|       0| 0|      1.027|     10.538
RTD|      1.070|      2.075|      5.694|       0| 0|      1.027|     10.538
RTD|      1.140|      2.074|      7.459|       0| 0|      1.027|     10.538
RTD|      1.135|      2.225|      9.373|       0| 0|      1.027|     10.538
RTD|      1.168|      2.081|      6.486|       0| 0|      1.027|     10.538
RTD|      1.062|      2.106|      9.617|       0| 0|      1.027|     10.538
RTD|      1.127|      2.161|      9.284|       0| 0|      1.027|     10.538
RTD|      1.079|      2.092|      5.480|       0| 0|      1.027|     10.538