Build Lightweight Python Components
Lightweight Python components do not require you to build a new container image for every code change. They’re intended for fast iteration in a notebook environment.
Advantages over container components:
- Faster iteration: No need to build new container image after every change (building images takes some time).
- Easier authoring: Components can be created in a local environment. Docker and Kubernetes are not required.
Building a lightweight Python component
To build a component, define a stand-alone Python function and then call
kfp.components.func_to_container_op(func)
to convert the function to a
component that can be used in a pipeline.
There are several requirements for the component function:
-
The function must be stand-alone.
- It should not use any code declared outside the function definition.
- Any imports should be added inside the main component function.
- Any helper functions should also be defined inside the main component function.
-
The function can only import packages that are available in the base image.
- If you need to import a package that’s not available in the default base
image you can try to find a container image that already includes the
required packages. (As a workaround you can use the
subprocess
module to runpip install
for the required package.)
- If you need to import a package that’s not available in the default base
image you can try to find a container image that already includes the
required packages. (As a workaround you can use the
-
If the function operates on numbers, the parameters must have type hints. Supported types are
int
,float
,bool
. All other arguments are passed as strings. -
To build a component with multiple output values, use Python’s typing.NamedTuple type hint syntax:
NamedTuple('MyFunctionOutputs', [('output_name_1', type), ('output_name_2', float)])
The
NamedTuple
class is already imported, so that it can be used in the function declaration.
Tutorials
See the notebook on lightweight Python component basics for an example of creating lightweight Python components and using them in a pipeline.
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.