Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SPM: Add SPMC documentation #107

Merged
merged 1 commit into from
May 18, 2021
Merged

Conversation

jellesels-arm
Copy link
Contributor

Add documentation for the following SPMC features:
-Handling FF-A messages
-SP initialisation and start
-Rxtx buffers

Signed-off-by: Jelle Sels [email protected]

Secure Partitions (SPs) are the endpoints used in the FF-A protocol. When
OP-TEE is used as a SPMC SPs run primarily inside S-EL0.

The SPMC also exposes the OP-TEE kernel as an secure endpoint itself. This is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/the OP-TEE kernel/OP-TEE core, privileged mode,/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

OP-TEE is used as a SPMC SPs run primarily inside S-EL0.

The SPMC also exposes the OP-TEE kernel as an secure endpoint itself. This is
used to handle all OP-TEE legacy operations. All legacy operations will be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be a bit early to call the communication with TAs legacy operations. I'm not sure if SPs will ever be able to replace TAs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jenswi-linaro agreed, "TAs" in this context means the whole GlobalPlaform programming model...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

when the SPMD is enabled.

SPs run from the initial boot of the system until power down and don't have any
build-in session management compared to user TA. There only means of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/There/The/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

``Early TSs``. This is done because SP have to be loaded during boot. Early TSs
are embedded inside the OP-TEE image. This makes it possible for us to load SPs
during the initialisation of OP-TEE and we are not restricted to as rich-os in
the normal world. LDELF is used to load the SPs into memory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/LDELF/Ldelf/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

The implementation of the SP handling in the SPMC is build upon OP-TEEs
``Early TSs``. This is done because SP have to be loaded during boot. Early TSs
are embedded inside the OP-TEE image. This makes it possible for us to load SPs
during the initialisation of OP-TEE and we are not restricted to as rich-os in
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sentence needs to be rephrased a little.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done



Each SP is loaded into the system using ldelf and started. This is done using
LDELF and is based around the same process as loading the early TAs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/LDELF/ldelf/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Each SP is loaded into the system using ldelf and started. This is done using
LDELF and is based around the same process as loading the early TAs.
All SPs are run after they are loaded and run until a FFA_MSG_WAIT is send by
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to put all FF-A defines inside ``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

skinparam backgroundcolor transparent
participant None_secure_world

box "S-El3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S-EL3

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Running and exiting SPs
-----------------------

THE SPMC resumes/starts q SP by calling the sp_enter(). This will set-up the SP
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

q?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@jforissier jforissier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not too sure about the typographic conventions for structs, function names and file names but I think they should all be in double back ticks.

SPMC
====

This document describes the SPMC(S-EL1) implementation for OP-TEE.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing space SPMC (S-EL1)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

OP-TEE is used as a SPMC SPs run primarily inside S-EL0.

The SPMC also exposes the OP-TEE kernel as an secure endpoint itself. This is
used to handle all OP-TEE legacy operations. All legacy operations will be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jenswi-linaro agreed, "TAs" in this context means the whole GlobalPlaform programming model...

encapsulated inside FF-A messages. The OP-TEE endpoint will unpack the messages
and afterwards handle them as standard OP-TEE calls. This is needed as
TF-A (S-EL3) does only allow FF-A messages to be passed to the secure world
when the SPMD is enabled.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this introduces a noticeable overhead in the handling of TA messages? (if so it could be an obstacle to the adoption of the new FF-A protocol).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the only overhead is in how shared memory is registered. In the OP-TEE message model it can be a one-step process for temporary shared memory, while with FF-A it's always a two-step process (first register with SPMC and then invoke OP-TEE). It's covered at page 15 in https://static.linaro.org/connect/lvc21/presentations/lvc21-305.pdf

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, good. Thanks.


.. code-block:: Make

OPTEE_OS_COMMON_FLAGS += CCFG_WITH_SPMC=y CFG_CORE_FFA=y
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CFG_WITH_SPMC

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

conf.py Outdated
@@ -41,7 +41,10 @@
extensions = [
'sphinx.ext.todo',
'sphinx.ext.viewcode',
'sphinx.ext.graphviz'
'sphinx.ext.graphviz',
#For plantuml to work you need a plantuml command be on your path. This
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing space # For...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

s to the Normal World buffers.


Structs and global objects
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this section is useful at all. It is implementation details that will likely become outdated and which would be much easier documented and understood in the code itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Secure Partitions (Loading the Image, Setting up the stack, HEAP, ...).
- Routing messages between endpoints:
The SPMC is responsible for passing FF-A messages from The Normal World
to SPs and back. It also needs to to pass FF-A messages between SPs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

The FF-A message is passed to the kernel thread and it will call the
sp_msg_handler function.

Whenever the SPMC (sp_msg_handler) receives a message not intended for one of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't all function names be in double back ticks and have parentheses?

``sp_msg_handler()``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

the SPs, it will exit the kernel thread and return to the Normal World passing
the FF-A message.

Currently only a FFA_MSG_SEND_DIRECT_REQ can be passed from the Normal World to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

``FFA_MSG_SEND_DIRECT_REQ``

Copy link
Contributor Author

@jellesels-arm jellesels-arm Mar 31, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Transmit and receive buffers are used by the FF-A mechanism to communicate
larger data structure between the SPMC and an endpoint. To accomplish this, each
SP has his own rxtx_buf for access to it's transmit and receive buffers. There
is also 1 defined for the Normal World (nw_buf in thread_spmc.c).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one

``nw_buf``
``thread_spmc.c``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted

conf.py Outdated
'sphinx.ext.graphviz',
#For plantuml to work you need a plantuml command be on your path. This
# shall start plantuml.jar as: java -jar /path/to/plantuml.jar $@
'sphinxcontrib.plantuml'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure how this will work, I've tried Google it, but didn't find any obvious solution for
a) Enable it in Travis
b) Enable it at readthedocs.io

There were some things that seems to be worth trying.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,

there are the following dependencies:

  1. graphviz
  2. java runtime environment
  3. plamtUML
  4. the sphinxcontrib-plantuml PyPI package

On Ubuntu 1-3 can be installed with sudo apt install plantuml . 4 can be installed with pip3 install sphinxcontrib-plantuml.

RTD

The RTD build container has planuml and its dependencies installed.
Additional python packages PyPI packages can be set-up with a configuration file, but TS documentation is using plantuml, build fine and we don't have the mentioned extra config. Although we have a requirements.txt file, which might be taken up by RTD automatically. This is not mentioned in the documentation though.

Travis

I don't know much about the environment used here. sudo apt install plantuml and pip install sphinxcontrib-plantuml should get everything needed, if the environment is ubuntu based.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gyuri-szing: thanks, that gives a couple of things to try. Having said that, I'm unable to do it right away, so the PR will have to be on hold for a little longer until I've found some time for try it out. Shouldn't be that long though ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbech-linaro did you had any time to look at this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't had time to look into it yet. Probably have to push it out a bit longer (if no-one else in @OP-TEE/approvers have some bandwidth to spare).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give it a shot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll give it a shot.

Thanks Jens!

end box

autoactivate on
None_secure_world -> SPMD: FFA_MSG_SEND_DIRECT_REQ \n <font color=red>SMC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Non-secure world" instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't have space in the participant name but I used the 'as' to change the tittle of the participant to "Non-secure world"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks


autoactivate on
None_secure_world -> SPMD: FFA_MSG_SEND_DIRECT_REQ \n <font color=red>SMC
SPMD -> thread_spmc: thread_spmc_msg_recv \n <font color=red>ERET
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably have () after each function name to help identifying functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jellesels-arm jellesels-arm force-pushed the spmc_docs branch 2 times, most recently from dae6508 to fd82c6d Compare April 6, 2021 08:02

The SPMC also exposes the OP-TEE core, privileged mode, as an secure endpoint
itself. This is used to handle all GlobalPlaform programming mode operations
All s will GlobalPlatform messages are encapsulated inside FF-A messages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Starting SPs
------------
SP are loaded and started as the last step in OP-TEE's initialisation process.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SPs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

:``sp_init_set_registers()``: Initialise the registers of the SP
:``sp_msg_handler()``: Handle the SPs FF-A message

Once all SP are loaded and started we return to the SPMD and the Normal World is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SPs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

is a SP.

When a ``FFA_MSG_SEND_DIRECT_REQ`` message is received by the spmc from the
Normal World, a new kernel thread is started.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can drop "kernel". Generally we avoid using this word for OP-TEE core since it's easily confused with the Linux kernel.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Running and exiting SPs
-----------------------

THE SPMC resumes/starts the SP by calling the sp_enter(). This will set-up the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

====

This document describes the SPMC (S-EL1) implementation for OP-TEE.
More information on the SPMC can be found in the FF-A specification can be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove 2nd occurrence of "can be found in the":

-More information on the SPMC can be found in the FF-A specification can be
-found int the
+More information on the SPMC can be found in the FF-A specification:
 `FF-A spec <https://developer.arm.com/documentation/den0077/latest>`_.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

The SPMC component is responsible for initialisation of the
Secure Partitions (Loading the Image, Setting up the stack, HEAP, ...).
- Routing messages between endpoints:
The SPMC is responsible for passing FF-A messages from The Normal World
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/The Normal World/normal world/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Secure Partitions (Loading the Image, Setting up the stack, HEAP, ...).
- Routing messages between endpoints:
The SPMC is responsible for passing FF-A messages from The Normal World
to SPs and back. It also needs to to pass FF-A messages between SPs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/It also needs to to pass/It is also responsible for passing/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

to SPs and back. It also needs to to pass FF-A messages between SPs.
- Memory management:
The SPMC is responsible for the memory management of the SPs. Memory
can be shared between SPs and from a SP to the Normal World.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/from a SP to the Normal World./between a SP and the normal world./

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

OP-TEE is used as a SPMC SPs run primarily inside S-EL0.

The SPMC also exposes the OP-TEE core, privileged mode, as an secure endpoint
itself. This is used to handle all GlobalPlaform programming mode operations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a dot . and end of the above line.

Is the sentence below true? I understand it is a goal but I think this is not yet implemented. If so, maybe this doc should not be stated that as something that is available.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was implemented by @jenswi-linaro

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

the source or the destination
is a SP.

When a ``FFA_MSG_SEND_DIRECT_REQ`` message is received by the spmc from the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/spmc/SPMC/g

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Every message received by the SPMC from the Normal World is handled in the
``thread_spmc_msg_recv()`` function.

When entering a SPs we need to be running on a thread. This is needed to be
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/SPs/SP/
s/on a thread/in an OP-TEE thread/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

``sp_enter()`` will copy the FF-A arguments and resume the SP.

When the SPMC needs to have access to the SPs memory, it will call
``ts_push_current_session()`` to gain access and ts_pop_current_session to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ts_pop_current_session formatting (s/ts_pop_current_session/``ts_pop_current_session``/)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

The SPMC resumes/starts the SP by calling the sp_enter(). This will set-up the
SP context and jump into S-EL0.
Whenever the SP performs a system call it will end-up in ``sp_handle_svc()``.
``sp_handle_svc()`` store the current context of the SP and makes sure that we
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/store/stores/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

OPTEE_OS_COMMON_FLAGS += CFG_WITH_SPMC=y CFG_CORE_FFA=y
OPTEE_OS_COMMON_FLAGS += CFG_WITH_SP=y
OPTEE_OS_COMMON_FLAGS += SP_PATHS="SP.elf"
TF_A_FLAGS += SPD=spmd SPMD_SPM_AT_SEL2=0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OPTEE_OS_COMMON_FLAGS relates to a config from OP-TEE/build.git. Maybe worth mentionning that here: or simply state that OP-TEE configuration should have:
CFG_WITH_SPMC=y
CFG_CORE_FFA=y
CFG_WITH_SP=y
SP_PATHS="path/to/sp-xxx.elf path/to/sp-yyy.elf

(by the way, it would be more consistent to rename SP_PATHS to CFG_SP_PATHS in optee_os.git)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose SP_PATHS follows EARLY_TA_PATHS

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true :) both should be CFG_s IMO. But this is probably not a real issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true :) both should be CFG_s IMO. But this is probably not a real issue.

No, it is on purpose that they are not CFG_* in order to make it possible to not build the "TA dev kit" twice, see https://github.com/OP-TEE/optee_os/blob/3.12.0/mk/config.mk#L297-L303 (indeed, defining a new CFG_ would cause recompilation). Probably not very important though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, fine. i had forgotten this. i found back your comment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OPTEE_OS_COMMON_FLAGS are still there

@jenswi-linaro
Copy link
Contributor

Please rebase this now that we have enabled plantuml support in travis and readthedocs.

SPMC Responsibilities
=====================

The SPMC is a critical component in the FF-A flow. Some of it mayor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/it mayor/its major/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

FF-A messages to be passed to the secure world when the SPMD is enabled.

SPs run from the initial boot of the system until power down and don't have any
build-in session management compared to GPD TEE TAs. The only means of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/build-in/built-in/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


SPs run from the initial boot of the system until power down and don't have any
build-in session management compared to GPD TEE TAs. The only means of
communicating with the outside world is through the FF-A spec defined messages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...through messages defined in the FF-A specification.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

SPs run from the initial boot of the system until power down and don't have any
build-in session management compared to GPD TEE TAs. The only means of
communicating with the outside world is through the FF-A spec defined messages.
SPs context is saved between executions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The context of a SP is saved between executions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


The
`Trusted Service <https://www.trustedfirmware.org/projects/trusted-services/>`_
repository includes the libSP libary which export all needed functions to build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really capital letters in libSP? libraries usually have lower case letters only.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done



Each SP is loaded into the system using ldelf and started. This is done using
Ldelf and is based around the same process as loading the early TAs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mentioning ldelf here again is a bit repetitive. You may want to rephrase these two sentences.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


The SPMC is split into 2 main message handlers:

:``thread_spmc_msg_recv()`` thread_spm.c: Used to handle message coming
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thread_spmc.c

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Every message received by the SPMC from the Normal World is handled in the
``thread_spmc_msg_recv()`` function.

When entering a SP we need to be running in a OP_TEE thread. This is needed to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/OP_TEE/OP-TEE/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Running and exiting SPs
-----------------------

The SPMC resumes/starts the SP by calling the sp_enter(). This will set-up the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sp_enter()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/set-up/set up/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


The SPMC resumes/starts the SP by calling the sp_enter(). This will set-up the
SP context and jump into S-EL0.
Whenever the SP performs a system call it will end-up in ``sp_handle_svc()``.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

end up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jellesels-arm jellesels-arm force-pushed the spmc_docs branch 2 times, most recently from ba144e2 to 4306977 Compare May 12, 2021 08:10
@jenswi-linaro
Copy link
Contributor

Reviewed-by: Jens Wiklander <[email protected]>

@jellesels-arm
Copy link
Contributor Author

@jforissier, @etienne-lms any more comments?

Copy link
Contributor

@jforissier jforissier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jellesels-arm please see below, thanks.

SPMC Responsibilities
=====================

The SPMC is a critical component in the FF-A flow. Some of its mayor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

major

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


- Initialisation and run-time management of the SPs:
The SPMC component is responsible for initialisation of the
Secure Partitions (Loading the Image, Setting up the stack, HEAP, ...).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • secure partitions (no initial capitalization)? This is arguable, like normal world / Normal World. What is the convention used in Arm documentation?
  • loading, image, setting, heap

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec uses Secure Partitions


The SPMC also exposes the OP-TEE core, privileged mode, as an secure endpoint
itself. This is used to handle all GlobalPlaform programming mode operations.
All GlobalPlatform messages are encapsulated inside FF-A messages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is assuming that some config option is set to y. Which one is it? CFG_WITH_SPMC? CFG_CORE_SEL1_SPMC? CFG_CORE_FFA? This needs to be clearly documented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


SPMC Program Flow
=================
SP images are stored OP-TEE image as early TAs are: the binary images are
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the OP-TEE image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

World.


Each SP is loaded into the system using ldelf and started. This is based around
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

``ldelf``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


Each SP is loaded into the system using ldelf and started. This is done using
Ldelf and is based around the same process as loading the early TAs.
All SPs are run after they are loaded and run until a ``FFA_MSG_WAIT`` is send
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still see "send"

embedded in OP-TEE Core in a read-only data section.
This makes it possible to load SPs during boot. Early TSs are embedded inside
the OP-TEE image. This makes it possible to load SPs during the initialisation
of OP-TEE and no rich-os is needed in the normal world. Ldelf is used to load
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • You are saying the same thing twice, aren't you?
  • ``ldelf``

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

OPTEE_OS_COMMON_FLAGS += CFG_WITH_SPMC=y CFG_CORE_FFA=y
OPTEE_OS_COMMON_FLAGS += CFG_WITH_SP=y
OPTEE_OS_COMMON_FLAGS += SP_PATHS="SP.elf"
TF_A_FLAGS += SPD=spmd SPMD_SPM_AT_SEL2=0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OPTEE_OS_COMMON_FLAGS are still there

of OP-TEE and no rich-os is needed in the normal world. Ldelf is used to load
and initialise the SPs.
This makes it possible to load SPs during boot and no rich-os is needed in the
normal world. ``Ldelf`` is used to load and initialise the SPs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lowercase el

TF_A_FLAGS += SPD=spmd SPMD_SPM_AT_SEL2=0
OPTEE_OS_COMMON_FLAGS += CFG_CORE_FFA=y #Enable the FF-A transport layer
OPTEE_OS_COMMON_FLAGS += SP_PATHS="path/to/sp-xxx.elf path/to/sp-yyy.elf" #Add the SPs to the OP-TEE image
TF_A_FLAGS += SPD=spmd SPMD_SPM_AT_SEL2=0 #Build TF-A with the SPMD enabled and without S-EL2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One space after the hash sign please

@jforissier
Copy link
Contributor

With my last two comments addressed:
Reviewed-by: Jerome Forissier <[email protected]>

Add documentation for the following SPMC features:
-Handling FF-A messages
-SP initialisation and start
-Rxtx buffers

Change-Id: If4a50787036119e30a8cdd92f35a7032aacec73f
Signed-off-by: Jelle Sels <[email protected]>
Reviewed-by: Jens Wiklander <[email protected]>
Reviewed-by: Jerome Forissier <[email protected]>
@jbech-linaro
Copy link
Contributor

Thanks @jellesels-arm for the contribution and taking the time to do this. Thanks Jens & Jerome for doing the review. Merging now!

@jbech-linaro jbech-linaro merged commit 3ef999a into OP-TEE:master May 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants