Enabling Core Isolation

Before performing dynamic core isolation, ensure that intelligent computing tasks and other general-purpose computing tasks have been started. Currently, only intelligent computing tasks that use vLLM as the inference engine are supported.

1. Obtain the AIHost-turbo code.

   1 2 3

   git clone https://gitcode.com/openeuler/AIHost-turbo.git cd AIHost-turbo/a-sched pip install .

2. Execute the affinity script. The following commands are for reference:

   1	sudo python examples/affinity_vllm.py -dp-size 4 -dp-size-local 4 -dp-start-rank 0 -tp-size 4 -ep -r

   After the execution is complete, the vLLM inference service can be executed in affinity mode. This isolates core resources of intelligent computing tasks from those of general-purpose computing tasks, thereby improving inference performance.

   Table 1 Script execution parameters

   Parameter	Description
   -r/--run	Runs affinity scheduling.
   -d/--dry-run	Performs trial run, which is used only for solution decision-making and does not affect the actual execution result.
   -p/--print	Prints the current affinity information.
   -restore	Restores the affinity to the state before scheduling.
   --tp-size	(Mandatory) Indicates the tensor parallel (TP) size. The value of this parameter is the same as that used during vLLM startup.
   --dp-size	(Mandatory) Indicates the data parallel (DP) size. The value of this parameter is the same as that used during vLLM startup.
   --dp-size-local	Indicates the DP size of the current node. (For a single node, set this parameter to the value of --dp-size.)
   --dp-start-rank	Indicates the DP start rank of the current node. (For a single node, set this parameter to 0.)
   -ep/--enable_ep	Enables expert parallelism (EP).

   A command output example is as follows:

   

   Command output explanation:

   • Process/Thread entry: describes the resource allocation range of a main computing process or an internal thread.

     1	- PROCESS[2024621]: name=VLLM::EngineCore_DP0, priority=NORMAL, socket=[0, 1, 2, 3], numa=[0-7], cluster=[0-79], cpu=[0-319]

     Table 2 Output parameter description

     Field	Example Value	Description
     Type	[PROCESS] / [Thread]	Indicates whether the entry is a process or a thread.
     PID	2024621	Indicates the unique identifier allocated by the OS, that is, process ID.
     Name	VLLM::EngineCore_DP0	Indicates the process name. DP0 usually indicates DP group 0. TP indicates tensor parallelism. EP indicates expert parallelism.
     Priority	NORMAL	Indicates the scheduling priority of a process or thread.
     Socket	[0, 1, 2, 3]	Indicates the IDs of the physical CPU sockets that can run the processes and threads.
     NUMA	[0-7]	Indicates the range of the NUMA nodes that can be accessed.
     CPU	[0-319]	Indicates the range of logical CPU cores that can run the processes and threads.

   • Hardware interrupt entry: describes the specific binding position of a hardware interrupt request (IRQ). It is the key to performance tuning and determines which CPU core is responsible for processing NPU communication.

     1	- Irq[2062](sq_send_trigger_irq): numa=[6], cluster=[62], cpu=[249]

     Table 3 Output parameter description

     Field	Example Value	Description
     Type	Irq[2062]	Identifies an interrupt request, followed by the interrupt number.
     Description	(sq_send_trigger_irq)	Indicates the interrupt type name, which usually represents a specific hardware communication event.
     NUMA	[6]	Indicates the specific NUMA node to which the interrupt is bound.
     Cluster	[62]	Indicates the ID of the cluster (logical group) to which the logical core belongs.
     CPU	[249]	Indicates the ID of the CPU core to which the interrupt is precisely bound. This is the most critical field.

   

   • The root permission is required for AIHost-turbo to modify the process/thread affinity. You need to explicitly add sudo or use the root user to run the command.
   • After affinity is enabled, a JSON file is generated. This file records the information used before affinity is enabled. If the file is deleted, script-based affinity state restoration cannot be performed.
3. Restores the affinity to the state before scheduling.

   1	sudo python examples/affinity_vllm.py -dp-size 4 -dp-size-local 4 -dp-start-rank 0 -tp-size 4 -ep -restore

   A command output example is as follows:

   

   

   When restoring the state, you still need to enter the dp-size and tp-size parameters used during startup. Otherwise, the processes and threads to be restored cannot be identified.