CS 518 - ProjectApple OS-X - Project: Grouping Threads by Affinity to a Particular Core based on Cache.12 December 2008 by Jean-Paul Lauren

Problem to be addressed

How can we group certain processes on one CPU?

{to take advantage of shared cache and optimizations by staying resident on one CPU}

Simple Approach

Add a rule to the scheduler, telling it to group based on a tag or token on a process.

Add this tag or token, hardcode it first – then later use a system call to create the value of a processor.

Simple Approach

Pretty staraight up approach that avoids directly modifying the scheduler.

Instead, add a rule to the scheduler that takes a token value and groups the processes based on that.

Detailed Approach

Proposed Solution

The Mach thread policy calls are defined in header /usr/include/mach/thread_policy.h and in /System/Library/Frameworks/System.framework/PrivateHeaders/mach/thread_policy.h as follows:

Existing Code - thread_policy.h

kern_return_t thread_policy_set(

thread_t thread,

thread_policy_flavor_t flavor,

thread_policy_t policy_info,

mach_msg_type_number_t count);

thread_policy.h

kern_return_t thread_policy_get(

thread_t thread,

thread_policy_flavor_t flavor,

thread_policy_t policy_info,

mach_msg_type_number_t *count);

boolean_t *get_default);

Modified Code - by adding a new flavor of thread policy:

#define THREAD_AFFINITY_POLICY 4

struct thread_affinity_policy {

integer_t affinity_tag;

};

by adding a new flavor of thread policy:

typedef struct thread_affinity_policy thread_affinity_policy_data_t;

typedef struct thread_affinity_policy *thread_affinity_policy_t;

by adding a new flavor of thread policy:

#define THREAD_AFFINITY_POLICY_COUNT ((mach_msg_type_number_t) \

(sizeof (thread_affinity_policy_data_t) / sizeof (integer_t)))

#define THREAD_AFFINITY_TAG_NULL 0

Tag or Token for grouping

By setting a (non-null) affinity tag for a thread, the thread is placed into the affinity set identified by the "tag". By default, all threads have the THREAD_AFFINITY_NULL affinity. A non-null tag is arbitrary and can convey application-specific information.

THREAD_AFFINITY_POLICY

The thread_policy_set (THREAD_AFFINITY_POLICY) call can be made after creating a thread but before starting it running in order to influence its initial placement.

Rule places processes according to available CPUs

For example, an application wanting to run 2 threads on separate L2 caches would set the threads with different affinity tags. On a dual core machine, this affinity will effectively be ignored. However, on a 4-core MacPro, the scheduler will try to run threads on separate packages. Similarly, on an 8-core MacPro, the scheduler will try to run these threads on separate dies (which may or may not be in the same physical CPU package).

Tag or Token

Threads with default affinity policy will be scheduled more freely on any processor. These threads will be preferentially migrated to run on an idle processor. Threads with affinity tags will tend to remain in place.

task_info.h

In addition, the header /usr/include/mach/task_info.h includes the following new information request:

struct task_affinity_tag_info {

integer_t count;

integer_t min;

integer_t max;

integer_t task_count;

};

task_info.h

typedef struct task_affinity_tag_info task_affinity_tag_info_data_t;

typedef struct task_affinity_tag_info *task_affinity_tag_info_t;

task_info.h

#define TASK_AFFINITY_TAG_INFO 16

#define TASK_AFFINITY_TAG_INFO_COUNT \

(sizeof(task_affinity_tag_info_data_t) / sizeof(natural_t))

This enables you to obtain the number (count) and range [min .. max] of the affinity tags currently defined for a task.