1# Compilation and Building Guide
2
3## Overview
4
5The OpenHarmony Compilation and Building subsystem provides a build framework based on Generate Ninja (GN) and Ninja. It abstracts the build and configuration granularity by component, and provides function enhancement for built-in modules and function extension for service modules. The OpenHarmony Compilation and Building subsystem provides the following functions:
6
7- Allows products to be assembled and independently built by component.
8- Supports version builds for mini, small, and standard systems and build of the software development kit (SDK), which facilitates application developers to use the Integrated development environment (IDE).
9- Supports customization and independent build based on different chipset solutions.
10
11### Application Scope
12
13This guide is applicable to mini, small, and standard systems. The [Chipset Solution](subsys-build-chip_solution.md) applies only to mini and small systems.
14
15### Basic Concepts
16
17
18Learn the following basic concepts before you get started:
19
20- Platform
21
22  A platform consists of a development board and the kernel. The subsystems and components vary with the platform.
23
24- Product
25
26  A product is a collection of components. The product image built can run on different development boards.
27
28- Subsystem
29
30  OpenHarmony is designed with a layered architecture, which consists of the kernel layer, system service layer, framework layer, and application layer from bottom up. For details, see [OpenHarmony Technical Architecture](https://gitee.com/openharmony#technical-architecture). System functions are built from components, subsystems, and then to the system. In a multi-device deployment, you can customize subsystems and components as required. A subsystem, as a logical concept, consists of the least required components.
31
32- Component
33
34  A component is a reusable software binary unit that contains source code, configuration files, resource files, and build scripts. It can be built independently, integrated in binary mode, and tested independently. The chipset solution mentioned in this guide is a special component in nature.
35
36- Module
37
38  A module is a target to build. A component can also be a target to build.
39
40- Feature
41
42  A component can provide differentiated functions through features.
43
44- GN
45
46  GN is used to generate Ninja files.
47
48- Ninja
49
50  Ninja is a small high-speed build system.
51
52- hb
53
54  hb is an OpenHarmony command line tool used to execute build commands.
55
56The Compilation and Build subsystem implements compilation and packaging through configuration. The following describes the relationships between the product, subsystem, component, and module.
57
58**Figure 1** Relationships between product, subsystem, component, and module
59
60![](figures/product_subsystem_component_module_relationships.png)
61
62The relationships between the product, subsystem, component, and module are as follows:
63
64- A subsystem is a collection of all components. A component can belong to only one subsystem.
65- A component is a collection of modules. A module can belong to only one component.
66- The product configuration file contains the configuration of all the components of the product. The component configuration can be reused.
67- A component used in different products can provide differentiated functions through variants or features.
68- A module is a target to build, and a component can also be a target to build.
69
70### Working Principles
71
72A product, component, and module can be built, but a subsystem cannot. The figure below shows the build process.
73
74**Figure 2** Build process
75
76![Build process](figures/build_process.png)
77
781. Run **hb set** to set the product to build.
79
802. Use **hb build** to build a product, development board, or component.
81
82   The procedure is as follows:
83
84   1. Read the **config.gni** file of the development board selected. The file contains the build toolchain, linking commands, and build options.
85   2. Run the **gn gen** command to read the product configuration and generate the **out** directory and **ninja** files for the product solution.
86   3. Run **ninja -C out/board/product** to start the build.
87   4. Package the files built, set the file attributes and permissions, and create a file system image.
88
89### Constraints
90
91Currently, only Ubuntu18.04 and Ubuntu20.04 are supported. Ubuntu22.04 is not supported.
92
93### Environment Configuration
94
95You must install the software packages required for build. The command is as follows:
96
97- Method 1: Run the script in the project directory.
98  ```shell
99  ./build/build_scripts/env_setup.sh
100  ```
101
102- Method 2: Run the **apt-get install** and **pip3 install** commands.
103  ```shell
104  apt-get update -y
105  apt-get install -y
106  # Install Python 3.9 for Ubuntu 20.04, and install Python 3.8 for Ubuntu 18.04.
107  apt-get install -y apt-utils binutils bison flex bc build-essential make mtd-utils gcc-arm-linux-gnueabi u-boot-tools python3.9 python3-pip git zip unzip curl wget gcc g++ ruby dosfstools mtools default-jre default-jdk scons python3-distutils perl openssl libssl-dev cpio git-lfs m4 ccache zlib1g-dev tar rsync liblz4-tool genext2fs binutils-dev device-tree-compiler e2fsprogs git-core gnupg gnutls-bin gperf lib32ncurses5-dev libffi-dev zlib* libelf-dev libx11-dev libgl1-mesa-dev lib32z1-dev xsltproc x11proto-core-dev libc6-dev-i386 libxml2-dev lib32z-dev libdwarf-dev
108  apt-get install -y grsync xxd libglib2.0-dev libpixman-1-dev kmod jfsutils reiserfsprogs xfsprogs squashfs-tools  pcmciautils quota ppp libtinfo-dev libtinfo5 libncurses5 libncurses5-dev libncursesw5 libstdc++6  gcc-arm-none-eabi vim ssh locales doxygen
109  apt-get install -y libxinerama-dev libxcursor-dev libxrandr-dev libxi-dev
110  # The following modules must be installed for Python. You can obtain the repo file from the source code of the build environment you use.
111  chmod +x /usr/bin/repo
112  pip3 install --trusted-host https://repo.huaweicloud.com -i https://repo.huaweicloud.com/repository/pypi/simple requests setuptools pymongo kconfiglib pycryptodome ecdsa ohos-build pyyaml prompt_toolkit==1.0.14 redis json2html yagmail python-jenkins
113  pip3 install esdk-obs-python --trusted-host pypi.org
114  pip3 install six --upgrade --ignore-installed six
115  #Install LLVM, hc-gen, gcc_riscv32, Ninja, node-v14.15.4-linux-x64, and GN. If the shell in use is not bash or zsh, configure the following environment variables:
116  # export PATH=/home/tools/llvm/bin:$PATH
117  # export PATH=/home/tools/hc-gen:$PATH
118  # export PATH=/home/tools/gcc_riscv32/bin:$PATH
119  # export PATH=/home/tools/ninja:$PATH
120  # export PATH=/home/tools/node-v12.20.0-linux-x64/bin:$PATH
121  # export PATH=/home/tools/gn:$PATH
122  # export PATH=~/.local/bin:$PATH
123  ```
124
125  >**NOTE**
126  >
127  >The hb tool will be installed during the installation of **ohos-build**. If hb tool fails to be installed, [install hb](../../device-dev/quick-start/quickstart-pkg-install-tool.md#hb-installation) again.
128
129
130## Configuration Rules
131
132To ensure that chipset and product solutions are decoupled from OpenHarmony, you need to follow certain rules during the configuration.
133
134- [Product Configuration Rules](subsys-build-product.md#configuration-rules)
135- [Subsystem Configuration Rules](subsys-build-subsystem.md#configuration-rules)
136- [Component Configuration Rules](subsys-build-component.md#configuration-rules)
137- [Module Configuration Rules](subsys-build-module.md#configuration-rules)
138- [Rust Module Configuration Rules](subsys-build-rust-compilation.md#configuration-rules)
139- [Chipset Solution Configuration Rules](subsys-build-chip_solution.md#configuration-rules)
140- [Feature Configuration Rules](subsys-build-feature.md#configuration-rules)
141- [System Capability Configuration Rules](subsys-build-syscap.md)
142
143## Guidelines
144
145### Directory Structure
146
147```shell
148
149/build                          # Directory for build
150
151├── __pycache__
152├── build_scripts/              # Python scripts for build
153├── common/
154├── config/                     # Build-related configuration
155├── core
156│   └── gn/                     # BUILD.gn configuration
157    └── build_scripts/
158├── docs
159gn_helpers.py*
160lite/                           # hb and preloader entry
161misc/
162├── ohos                        # Configuration of OpenHarmony build and packaging
163│   ├── kits                    # Kits build and packaging templates and processing
164│   ├── ndk                     # NDK templates and processing
165│   ├── notice                  # Notice templates and processing
166│   ├── packages                # Distribution packaging templates and processing
167│   ├── sa_profile              # SA profiles and processing
168│   ├── sdk                     # SDK templates and processing, which contains the module configuration in the SDK
169│   └── testfwk                 # Testing-related processing
170├── ohos.gni*                   # Common .gni files (facilitating one-time import of each module)
171├── ohos_system.prop
172├── ohos_var.gni*
173├── prebuilts_download.sh*
174├── print_python_deps.py*
175├── scripts/
176├── subsystem_config.json
177├── subsystem_config_example.json
178├── templates/                  # C/C++ build templates
179├── test.gni*
180├── toolchain                   # Build toolchain configuration
181├── tools                       # Common tools
182├── version.gni
183├── zip.py*
184
185```
186
187
188
189### Build Commands
190
191Run the **prebuilts** script in the root directory of the source code to perform precompilation and install the compiler and binary tool.
192
193```shell
194bash build/prebuilts_download.sh
195```
196
197Then, run the build commands in command line (CLI) mode or hb mode.
198
1991. Using the CLI
200
201   - Run the following command in the root directory of the source code to build a full distribution:
202
203     Release version:
204
205     ```shell
206     ./build.sh --product-name {product_name}
207     ```
208
209     Debug version:
210
211     ```shell
212     ./build.sh --product-name {product_name} --gn-args is_debug=true
213     ```
214
215     > **CAUTION**
216     >
217     > Due to the limitation of the image size, the full build for the debug version may fail to be burnt. You are advised to build the binary file for each module separately. Run the following command to build a module separately:
218     >
219     > ```
220     > ./build.sh --product-name {product_name} --gn-args is_debug=true --build-target {target_name}
221     > ```
222     >
223     > **{product_name}** specifies the product platform supported by the current distribution, for example, **hispark_taurus_standard**.
224     >
225     > The image generated is stored in the **out/{device_name}/packages/phone/images/** directory.
226
227   - The **./build.sh** command supports the following options:
228
229     ```shell
230     -h, --help                                       # Display help information and exit.
231     --source-root-dir=SOURCE_ROOT_DIR                # Specify the path.
232     --product-name=PRODUCT_NAME                      # Specify the product name.
233     --device-name=DEVICE_NAME                        # Specify the device name.
234     --target-cpu=TARGET_CPU                          # Specify the CPU.
235     --target-os=TARGET_OS                            # Specify the operating system.
236     -T BUILD_TARGET, --build-target=BUILD_TARGET     # Specify one or more targets to build.
237     --gn-args=GN_ARGS                                # Specify GN parameters.
238     --ninja-args=NINJA_ARGS                          # Specify Ninja parameters.
239     -v, --verbose                                    # Display all commands used.
240     --keep-ninja-going                               # Keep Ninja going until 1,000,000 jobs fail.
241     --jobs=JOBS
242     --export-para=EXPORT_PARA
243     --build-only-gn                                  # Perform GN parsing and do not run Ninja.
244     --ccache                                         # (Optional) Use ccache for build. You need to install ccache locally.
245     --fast-rebuild                                   # Specify whether to allow fast rebuild. The default value is False.
246     --log-level=LOG_LEVEL                            # Specify the log level used in the build process. The options are debug, info, and error. The default value is info.
247     --device-type=DEVICE_TYPE                        # Specify the device type. The default value is default.
248     --build-variant=BUILD_VARIANT                    #Specify the device operation mode. The default value is user.
249     ```
250
251
252
2532. Using the hb tool
254
255   **hb** is an OpenHarmony command line tool for executing build commands. Common hb commands are described as follows:
256
257   **hb set**
258
259   Sets the product to build.
260
261   ```shell
262   hb set -h
263   usage: hb set [-h] [-root [ROOT_PATH]] [-p]
264
265   optional arguments:
266     -h, --help            show this help message and exit
267     -root [ROOT_PATH], --root_path [ROOT_PATH]
268                           Set OHOS root path
269     -p, --product         Set OHOS board and kernel
270   ```
271
272   - If you run **hb set** with no argument, the default setting process starts.
273
274   - You can run **hb set -root** *dir* to set the root directory of the source code.
275
276   - You can run **hb set -p** to set the product to build.
277
278
279
280   **hb env**
281
282   Displays current settings.
283
284   ```shell
285   hb env
286   [OHOS INFO] root path: xxx
287   [OHOS INFO] board: hispark_taurus
288   [OHOS INFO] kernel: liteos
289   [OHOS INFO] product: ipcamera
290   [OHOS INFO] product path: xxx/vendor/hisilicon/ipcamera
291   [OHOS INFO] device path: xxx/device/hisilicon/hispark_taurus/sdk_linux_4.19
292   ```
293
294
295
296   **hb build**
297
298   Builds a product, component, module, or chipset solution.
299
300   ```shell
301   hb build -h
302   usage: hb build [-h] [-b BUILD_TYPE] [-c COMPILER] [-t [TEST [TEST ...]]] [-cpu TARGET_CPU] [--dmverity] [--tee]
303                   [-p PRODUCT] [-f] [-n] [-T [TARGET [TARGET ...]]] [-v] [-shs] [--patch] [--compact-mode]
304                   [--gn-args GN_ARGS] [--keep-ninja-going] [--build-only-gn] [--log-level LOG_LEVEL] [--fast-rebuild]
305                   [--device-type DEVICE_TYPE] [--build-variant BUILD_VARIANT]
306                   [component [component ...]]
307
308   positional arguments:
309     component             name of the component, mini/small only
310
311   optional arguments:
312     -h, --help            show this help message and exit
313     -b BUILD_TYPE, --build_type BUILD_TYPE
314                           release or debug version, mini/small only
315     -c COMPILER, --compiler COMPILER
316                           specify compiler, mini/small only
317     -t [TEST [TEST ...]], --test [TEST [TEST ...]]
318                           compile test suit
319     -cpu TARGET_CPU, --target-cpu TARGET_CPU
320                           select cpu
321     --dmverity            enable dmverity
322     --tee                 Enable tee
323     -p PRODUCT, --product PRODUCT
324                           build a specified product with {product_name}@{company}
325     -f, --full            full code compilation
326     -n, --ndk             compile ndk
327     -T [TARGET [TARGET ...]], --target [TARGET [TARGET ...]]
328                           compile single target
329     -v, --verbose         show all command lines while building
330     -shs, --sign_haps_by_server
331                           sign haps by server
332     --patch               apply product patch before compiling
333     --compact-mode        compatible with standard build system set to false if we use build.sh as build entrance
334     --gn-args GN_ARGS     specifies gn build arguments, eg: --gn-args="foo="bar" enable=true blah=7"
335     --keep-ninja-going    keeps ninja going until 1000000 jobs fail
336     --build-only-gn       only do gn parse, do not run ninja
337     --log-level LOG_LEVEL
338                           specifies the log level during compilationyou can select three levels: debug, info and error
339     --fast-rebuild        it will skip prepare, preloader, gn_gen steps so we can enable it only when there is no change
340                           for gn related script
341     --device-type DEVICE_TYPE
342                           specifies device type
343     --build-variant BUILD_VARIANT
344                           specifies device operating mode
345   ```
346
347   - If you run **hb build** with no argument, the previously configured code directory, product, and options are used for the build. The **-f** option deletes all products to be built. It is equivalent to running **hb clean** and **hb build**.
348
349   - You can run **hb build** *{component_name}* to build product components separately based on the development board and kernel set for the product, for example, **hb build kv_store**.
350
351   - You can run **hb build -p ipcamera@hisilicon** to skip the setting step and build the product directly.
352
353   - You can run **hb build** in **device/board/device_company** to select the kernel and build an image that contains the kernel and drivers only based on the current development board and the selected kernel.
354
355
356
357   **hb clean**
358
359   Deletes all the files except **args.gn** and **build.log** in the **out** directory (default). To clear files in a specified directory, add the directory parameter to the command, for example, **hb clean out/board/product**.
360
361   ```shell
362   hb clean
363   usage: hb clean [-h] [out_path]
364
365   positional arguments:
366     out_path    clean a specified path.
367
368   optional arguments:
369     -h, --help  show this help message and exit
370   ```
371
372   > **NOTE**
373   >
374   > - For details about how to set up the build environment and perform the build, see the related topics in [Getting Started](../quick-start/quickstart-overview.md).
375   > - OpenHarmony also provides the Docker environment, which spares the installation of the build tool. For details, see [Docker Environment](../get-code/gettools-acquire.md).
376
377### Building Procedures
378
379You can add and build a product, component, chipset solution, and module. For details, see:
380
381- [Adding and Building a Product](subsys-build-product.md#adding-and-building-a-product)
382- [Adding and Building a Component](subsys-build-component.md#adding-and-building-a-component)
383- [Adding and Building a Module](subsys-build-module.md#adding-and-building-a-module)
384- [Adding and Building a Chipset Solution](subsys-build-chip_solution.md#adding-and-building-a-chipset-solution)
385
386## FAQs
387
388- [Common Build Problems and Solutions](subsys-build-FAQ.md)
389
390## Reference
391
392- [deps and external_deps](subsys-build-reference.md#deps-and-external_deps)
393- [Using Sanitizer](subsys-build-reference.md#using-sanitizer)
394- [Information Collected by the Open Source Software Notice](subsys-build-reference.md#information-collected-by-the-open-source-software-notice)
395- [Parameters for Accelerating Local Build](subsys-build-reference.md#parameters-for-accelerating-local-build)
396- [Viewing Ninja Build Information](subsys-build-reference.md#viewing-ninja-build-information)
397- [Customizing the chip_prod Image](subsys-build-reference.md#customizing-the-chip_prod-image)
398
399