Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. (Semi-)permanently through an RTOS interfaces such as pthread_attr_setaffinity(), or
  2. Temporarily through new scheduling logic.

...

Tasks/threads

...

that

...

are

...

assigned

...

to

...

a

...

CPU

...

via

...

an

...

interface

...

like

...

pthread_attr_setaffinity()

...

would

...

never

...

go

...

into

...

the

...

g_readytorun

...

list,

...

but

...

would

...

only

...

go

...

into

...

the

...

g_assignedtasks

...

[n]

...

list

...

for

...

the

...

CPU

...

n

...

to

...

which

...

the

...

thread

...

has

...

been

...

assigned.

...

Hence,

...

the

...

g_readytorun

...

list

...

would

...

hold

...

only

...

unassigned

...

tasks/threads.

An indication within the TCB would indicated whether or not a task/thread is assigned to a CPU and, if so, which CPU it is assigned to.

Scheduling logic would temporarily assign a task or thread to a CPU. The assignment is only temporary because state data in the TCB would indicate that the task is unassigned when, hence, it could be returned to the g_readytorun list later.

...

The

...

assigned

...

tasks

...

lists

...

lists

...

would

...

be

...

prioritized.

...

The

...

highest

...

priority

...

task,

...

and

...

the

...

one

...

currently

...

executing

...

on

...

CPU

...

n

...

would

...

be

...

the

...

one

...

at

...

the

...

head

...

of

...

g_assignedtasks

...

[n]

...

.

...

Tasks

...

after

...

the

...

active

...

task

...

are

...

ready-to-run

...

and

...

assigned

...

to

...

this

...

CPU.

...

The

...

tail

...

of

...

this

...

assigned

...

task

...

list,

...

the

...

lowest

...

priority

...

task,

...

is

...

always

...

the

...

CPU's

...

IDLE

...

task.

...

The

...

CPU

...

n

...

scheduling

...

logic

...

would

...

execute

...

whenever

...

the

...

currently

...

running

...

task

...

is

...

removed

...

from

...

the

...

head

...

of

...

g_assignedtasks

...

[n]

...

. The

...

algorithm

...

might

...

be

...

something

...

like:

Code Block

  /* Is the assigned task list for the CPU empty? */
    
  if (g_assignedtasks[cpu].head == NULL)
    {
      /* No.. Is the task at the head of the assigned list for the CPU lower

...

     * in priority that the current (unassigned) task at the head of

...

 the
     * ready-to-run list?

...

Code Block
     */
  
    FAR struct tcb_s *rtcb = (FAR struct tcb_s *)g_readytorun.head ;
      FAR struct tcb_s *atcb = (FAR struct tcb_s *)g_assignedtasks[cpu].head;
      if (atcb->sched_priority < rtcb->sched_priority)
        {
          /* Remove the TCB from the head of the g_readytorun list. */
    
          /* Add that TCB to the g_assignedtasks[cpu] list (it will go at the
  • head of the list).
  • /
Code Block
        }
 * head of the list).
         */
      }
  
    /* Now activate the task at the head of the g_assignedtasks[cpu] list on

...

     * the CPU.

...

Code Block
     */
  
  }

The Current Task

There is a lot of logic in the RTOS now that obtains the TCB for the currently excuting task by examining the head of the g_readytorun list. You will see this assignment in many places, both in the core OS logic in nuttx/sched but also in architecture-specific logic under nuttx/arch.

...

unmigrated-wiki-markup
Code Block
    #define current_task(cpu)  ((FAR struct tcb_s *)g_readytorun.head)
    #define this_cpu()         (0)
    #define this_task()        (current_task(this_cpu))

Of

...

course,

...

that

...

would

...

not

...

work

...

with

...

the

...

proposed

...

changes.

...

We

...

would

...

need

...

to

...

then

...

get

...

the

...

TCB

...

of

...

the

...

currently

...

executing

...

task/thread

...

for

...

CPU

...

n

...

from

...

the

...

head

...

of

...

g_assignedtasks

...

[n]

...

. I would propose a replacing the above assignment with a macro like current_task()

...

where

...

that

...

macro

...

might

...

expand

...

to:

Code Block
    #ifdef CONFIG_SMP
    #  define current_task(cpu)  ((FAR struct tcb_s *)g_assignedtasks[cpu].head)
    #  define this_cpu()         up_cpu_index()
    #else
    #  define current_task(cpu)  ((FAR struct tcb_s *)g_readytorun.head)
    #  define this_cpu()         (0)
    #endif
    #define this_task()          (current_task(this_cpu))

where up_cpu_index() is some new MCU specific interface that will return an index associated with the currently active CPU.

...

NOTE

...

that

...

this

...

is

...

a

...

two

...

step

...

operations:

...

Step

...

1.

...

Get

...

the

...

CPU

...

number

...

and

...

Step

...

2:

...

Use

...

the

...

CPU

...

number

...

as

...

an

...

index

...

into

...

the

...

g_assignedtasks

...

[]

...

array

...

of

...

lists.

...

This

...

must

...

be

...

atomic!

...

The

...

schedule

...

should

...

be

...

locked

...

to

...

assure

...

that

...

the

...

task

...

is

...

not

...

suspended

...

after

...

fetching

...

the

...

CPU

...

number

...

then

...

restarted

...

on

...

a

...

different

...

CPU

...

to

...

access

...

the

...

g_assignedtasks

...

[]

...

array of

...

lists.

The IDLE Task

Without SMP, the g_readytorun list always ends with the TCB of IDLE task. It is always guaranteed to be at the end of the list because the list is prioritized and because the IDLE task has an impossibly low priority that no other task/thread could have. The IDLE task is necessary because it gives the CPU something to execute when there is nothing else to be done.

...

But

...

with

...

SMP,

...

there

...

are

...

multiple

...

CPUs

...

that

...

need

...

something

...

to

...

do

...

when

...

there

...

is

...

nothing

...

else

...

to

...

do.

...

I

...

am

...

tentatively

...

thinking

...

that

...

each

...

CPU

...

needs

...

its

...

own

...

IDLE

...

thread

...

whose

...

TCB

...

would

...

reside

...

at

...

the

...

end

...

of

...

each

...

g_assignedtasks

...

[cpu]

...

list.

...

But

...

that

...

does

...

feel

...

wasteful

...

to

...

me

...

(I

...

already

...

think

...

that

...

a

...

single

...

IDLE

...

thread

...

is

...

wasteful!).

I am not certain the mechanism as of this writing, but I assume that the nx_start() initialization logic would need to create an IDLE task for each CPU and assign each IDLE task to each CPU.

...

  • Special aligned stack allocation,unmigrated-wiki-markup
  • Logic

    to

    write

    the

    CPU

    index

    into

    the

    stack

    when

    each

    thread

    is

    \

    [re-]started.

This would also place an upper limit on the size of the stack: If we are going to find the far end of the stack by simply ANDing out the lower bits, then size of that mask would also determine the maximum size of the stack.

...

  • Keep the task data structures stable while they are being analyzed.
  • Find the lowest priority running task which could be on any CPU.
  • Wiki Markup

    If

    that

    priority

    is

    lower

    than

    the

    priority

    task,

    then

    replace

    it

    with

    the

    new

    task

    at

    the

    head

    of

    the

    {{

    g_assignedtasks

    \

    []

    }}

    list.

  • If not, find the task with the next lowest priority and compare that one.
  • Continue until until the new task is assigned to a CPU or until it is determined that all of the currently running tasks are higher priority than the new task. In that base, the new task should be added to the g_readytorun list.

...

Code Block
    int up_cpu_resume(int cpu);
Wiki Markup

Restart

...

the

...

CPU

...

with

...

the

...

task

...

at

...

the

...

head

...

of

...

the

...

g_assignedtasks

...

[]

...

 list

...

NOTE also the "Signal Handling" paragraph below. The same issue exists for dispatching signals to threads actively running on another CPU.

...

  • There is a global lock count g_cpu_lockset that includes a bit for each CPU: If the bit is '1', then the corresponding CPU has the scheduler locked; if '0', then the CPU does not have the scheduler locked.
  • Wiki Markup

    Scheduling

    logic

    would

    set

    the

    bit

    associated

    with

    the

    {{

    cpu

    }}

    in

    {{

    g_cpu_lockset

    }}

    when

    the

    TCB

    at

    the

    head

    of

    the

    {{

    g_assignedtasks

    \

    [cpu]

    }}

    list

    transitions

    has

    {{

    lockount

    >

    0

    }}

    .

    This

    might

    happen

    when

    {{

    sched_lock()

    }}

    is

    called,

    or

    after

    a

    context

    switch

    that

    changes

    the

    TCB

    at

    the

    head

    of

    the

    {{

    g_assignedtasks

    \

    [cpu]

    }}

    list.

    Wiki Markup
  • Similarly,

    the

    {{

    cpu

    }}

    bit

    in

    the

    global

    {{

    g_cpu_lockset

    }}

    would

    be

    cleared

    when

    the

    TCB

    at

    the

    head

    of

    the

    {{

    g_assignedtasks

    \

    [cpu]

    }} list has {{lockount

    list has lockount ==

    0

    }}

    .

    This

    might

    happen

    when

    {{

    sched_unlock()

    }}

    is

    called,

    or

    after

    a

    context

    switch

    that

    changes

    the

    TCB

    at

    the

    head

    of

    the

    {{

    g_assignedtasks

    \

    [cpu]

    }}

    list.

  • Modification of the global g_cpu_lockset must be protected by a simplified spinlock, g_cpu_schedlock. That spinlock would be taken when sched_lock() is called, and released when sched_unlock() is called. This assures that the scheduler does enforce the critical section. NOTE: Because of this spinlock, there should never be more than one bit set in g_cpu_lockset; attempts to set additional bits should be cause the CPU to block on the spinlock. However, additional bits could get set in 'g_cpu_lockset' due to the context switches on the various CPUs.unmigrated-wiki-markup
  • Each

    the

    time

    the

    head

    of

    a

    {{

    g_assignedtasks

    \

    [

    }}

    ]

    list

    changes

    and

    the

    scheduler

    modifies

    {{

    g_cpu_lockset

    }}

    ,

    it

    must

    also

    set

    {{

    g_cpu_schedlock

    }}

    depending

    on

    the

    new

    state

    of

    {{

    g_cpu_lockset

    }}

    .

  • Logic that currently uses the currently running tasks lockcount should instead use the global g_cpu_schedlock. A value of SP_UNLOCKED would mean that no CPU has pre-emption disabled; SP_LOCKED would mean that at least one CPU has pre-emption disabled.

...