Description jiangxin00 2023-10-23 19:52:13 UTC

Description of problem:

多CSV3虚拟机同时启动，需要并发从CMA分配内存，以提升多CSV3虚拟机并发启动性能。

关于CMA分配并发是否使用cma_mutex lock，主线上来回修改了多次。最近一次修改如下，把cma_mutex lock又加回来，原因是当cma内存较小（128MB)，而pageblock MAX_ORDER_NR_PAGES较大（32MB），测试用例中此时只有4个cma slot可用，如果没有cma_mutex lock，并发场景下将会导致dma_alloc_coherent失败。

而CSV3的CMA内存较大(1GB),而pageblock为4MB，此时一个cma area有256个cma slot可用，另外CSV3的cma area很多，如果预留16G，那么有16*256个cma slot可用。因此对CSV3的cma策略不需要cma_mutex lock。


author	Dong Aisheng <aisheng.dong@nxp.com>	2022-05-13 15:11:26 -0700
committer	Andrew Morton <akpm@linux-foundation.org>	2022-05-13 15:11:26 -0700
commit	60a60e32cf91169840abcb4a80f0b0df31708ba7 (patch)
tree	89c848f1817586e4e92ed92fb0b4517d9cd7cd8b
parent	9039b8335276569670caaf23606335946625e764 (diff)
download	linux-60a60e32cf91169840abcb4a80f0b0df31708ba7.tar.gz
Revert "mm/cma.c: remove redundant cma_mutex lock"
This reverts commit a4efc174b382fcdb which introduced a regression issue
that when there're multiple processes allocating dma memory in parallel by
calling dma_alloc_coherent(), it may fail sometimes as follows:

Error log:
cma: cma_alloc: linux,cma: alloc failed, req-size: 148 pages, ret: -16
cma: number of available pages:
3@125+20@172+12@236+4@380+32@736+17@2287+23@2473+20@36076+99@40477+108@40852+44@41108+20@41196+108@41364+108@41620+
108@42900+108@43156+483@44061+1763@45341+1440@47712+20@49324+20@49388+5076@49452+2304@55040+35@58141+20@58220+20@58284+
7188@58348+84@66220+7276@66452+227@74525+6371@75549=> 33161 free of 81920 total pages

The root cause of this issue is that since commit a4efc174b382 ("mm/cma.c:
remove redundant cma_mutex lock"), CMA supports concurrent memory
allocation.  It's possible that the memory range process A trying to alloc
has already been isolated by the allocation of process B during memory
migration.

The problem here is that the memory range isolated during one allocation
by start_isolate_page_range() could be much bigger than the real size we
want to alloc due to the range is aligned to MAX_ORDER_NR_PAGES.

Taking an ARMv7 platform with 1G memory as an example, when
MAX_ORDER_NR_PAGES is big (e.g.  32M with max_order 14) and CMA memory is
relatively small (e.g.  128M), there're only 4 MAX_ORDER slot, then it's
very easy that all CMA memory may have already been isolated by other
processes when one trying to allocate memory using dma_alloc_coherent(). 
Since current CMA code will only scan one time of whole available CMA
memory, then dma_alloc_coherent() may easy fail due to contention with
other processes.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: