Obtaining the Resource Usage of TAs in the TEE

File Access Permission

The CCOS grants TAs the permission to access the following files. The file formats are basically the same as those in the Linux man manual.

• /proc/stat provides statistics about the OS and each CPU core. You can learn about the status and performance indicators of each CPU. The format is as follows:

  cpu <user> <nice> <system> <idle> <iowait> <irq> <softirq> <steal> <guest> <guest_nice>
  cpu0 <user> <nice> <system> <idle> <iowait> <irq> <softirq> <steal> <guest> <guest_nice>
  ...
  cpuN <user> <nice> <system> <idle> <iowait> <irq> <softirq> <steal> <guest> <guest_nice>
  intr <total> <num> ...
  ctxt <num>
  btime <num>
  processes <num>
  procs_running <num>
  procs_blocked <num>
  softirq <total> <num> ...

  N indicates the number of CPU cores. This file provides information about the total CPU usage and the usage of each CPU core. For example, calculate the user-space CPU usage as follows:

  

  

  • Only the user, system, idle, and irq statistics are collected in the TEE OS, and statistics about other items are not collected.
  • By default, each CPU on the TEE side is bound to an smcd thread, which remains in a wait state when idle. Upon receiving an OpenSession request from a CA, the thread wakes up to execute the corresponding session task. Therefore, when the TEE OS is idle, all CPU cores run the smcd thread and the system usage is 100%. When a thread in the TA is bound to a CPU core, the user-space thread and the kernel-space smcd thread of the TA evenly share the CPU. That is, the user-space thread and kernel-space smcd thread each occupy 50% of the CPU. Before binding threads to CPU cores, run CPU_ISSET to check that the target CPU cores are online.
  • When a CA is bound to a CPU core, not all TAs run on the core. That is, the user-space CPU usage of the core is not 100%.
  • To obtain CPU usage in a multi-thread scenario, read the /proc/self/stat file before the service threads exit.
• /proc/uptime provides the system uptime and cumulative idle time since boot. The format is as follows:

  <uptime> <idle_time>

  The values are in seconds, whereas time values in most other /proc files are represented in clock ticks. The ratio between these two units is given by sysconf(_SC_CLK_TCK).

• /proc/loadavg provides information about the average system load (number of active tasks in the system in a recent period). The format is as follows:

  <1min_avg> <5min_avg> <15min_avg>

  By default, each CPU core in the TEE is bound to an smcd thread. Therefore, if no TA is running, the load is the number of CPU cores by default.

• /proc/self/* provides statistics and information about the current process. Two of the most commonly used files are introduced below:
  • The /proc/self/statm file provides memory usage information about the current process.

    <size> <resident> <shared> <text> <lib> <data> <dt>

    The unit is page. You can obtain the page size in the TEE using the sysconf(_SC_PAGESIZE) API. To obtain the total memory size, call the int sysinfo(struct sysinfo *info) API.

  • The /proc/self/stat file provides running statistics about the current process.

    <pid> (<comm>) <state> <ppid> <pgrp> <session> <tty_nr> <tpgid> <flags> <minflt> <cminflt> <majflt> <cmajflt> <utime> <stime> <cutime> <cstime> <priority> <nice> <num_threads> <itrealvalue> <starttime> <vsize> <rss> <rsslim> <startcode> <endcode> <startstack> <kstkesp> <kstkeip> <signal> <blocked> <sigignore> <sigcatch> <wchan> <nswap> <cnswap> <exit_signal> <processor> <rt_priority> <policy> <delayacct_blkio_ticks> <guest_time> <cguest_time>

    The time unit is measured in clock ticks. When combining this data with /proc/uptime to calculate the process CPU utilization, be sure to account for the difference in time units between them. Please refer to the formula below:

    

    The numerator represents the CPU time accumulated by the process within the measurement interval, which can be retrieved from /proc/self/stat. This is typically the sum of user time (utime) and system time (stime). Depending on your requirements, you can decide whether to include CPU times from other states. The denominator is the total system time within the measurement interval. sysconf(_SC_CLK_TCK) indicates the clock ticks per second.

Example of the TA's CPU usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

#define BUF_SIZE 1024
#define PROC_SELF_STAT "/proc/self/stat"
#define DELTA_TIME 1 // s

// Read the /proc file to obtain the original data.
int read_cpu_time(double *utime, double *stime) {
    int fd;
    char buf[BUF_SIZE];
    char *token;
    
    // Read the /proc/self/stat content.
    fd = open(PROC_SELF_STAT, O_RDONLY);
    if (fd == -1) {
        tloge("Failed to open %s\n", PROC_SELF_STAT);
        return -1;
    }
    if (read(fd, buf, BUF_SIZE) == -1) {
        tloge("Failed to read %s\n", PROC_SELF_STAT);
        close(fd);
    }
    close(fd);
    // Parse and obtain the utime and stime.
    token = strtok(buf, " ");
    for (int i = 1; i <= 17 && token != NULL; i++) {
        if (i == 14) *utime = atof(token);  // utime in jiffies
        if (i == 15) *stime = atof(token);  // stime in jiffies
        token = strtok(NULL, " ");
    }
    return 0;
}

// Calculate the CPU usage of a TA.
void calculate_cpu_usage() {
    double utime1, stime1;
    double utime2, stime2;
    double cpu_usage = 0;
    
    read_cpu_time(&utime1, &stime1);
    // or do other work
    sleep(DELTA_TIME);  
    read_cpu_time(&utime2, &stime2);

    cpu_usage = ((utime2 + stime2) - (utime1 + stime1)) / (DELTA_TIME * sysconf(_SC_CLK_TCK));
    tlogi("cpu usage is %lf %%\n", cpu_usage * 100);
}