android build system中product的继承(inherit-product),加载(import-products)和选择(lunch)

一、前言

android源码中有很多product,进行配置时,会将源码中所有product的信息都读进来(不用的product的信息也会被读进来),其中每个product,可以包含如下信息

#
# Functions for including product makefiles
#

_product_var_list := \
    PRODUCT_NAME \
    PRODUCT_MODEL \
    PRODUCT_LOCALES \
    PRODUCT_AAPT_CONFIG \
    PRODUCT_AAPT_PREF_CONFIG \
    PRODUCT_AAPT_PREBUILT_DPI \
    PRODUCT_PACKAGES \
    PRODUCT_PACKAGES_DEBUG \
    PRODUCT_PACKAGES_ENG \
    PRODUCT_PACKAGES_TESTS \
    PRODUCT_DEVICE \
    PRODUCT_MANUFACTURER \
    PRODUCT_BRAND \
    PRODUCT_PROPERTY_OVERRIDES \
    PRODUCT_DEFAULT_PROPERTY_OVERRIDES \
    PRODUCT_CHARACTERISTICS \
    PRODUCT_COPY_FILES \
    PRODUCT_OTA_PUBLIC_KEYS \
    PRODUCT_EXTRA_RECOVERY_KEYS \
    PRODUCT_PACKAGE_OVERLAYS \
    DEVICE_PACKAGE_OVERLAYS \
    PRODUCT_SDK_ATREE_FILES \
    PRODUCT_SDK_ADDON_NAME \
    PRODUCT_SDK_ADDON_COPY_FILES \
    PRODUCT_SDK_ADDON_COPY_MODULES \
    PRODUCT_SDK_ADDON_DOC_MODULES \
    PRODUCT_SDK_ADDON_SYS_IMG_SOURCE_PROP \
    PRODUCT_DEFAULT_WIFI_CHANNELS \
    PRODUCT_DEFAULT_DEV_CERTIFICATE \
    PRODUCT_RESTRICT_VENDOR_FILES \
    PRODUCT_VENDOR_KERNEL_HEADERS \
    PRODUCT_BOOT_JARS \
    PRODUCT_SUPPORTS_BOOT_SIGNER \
    PRODUCT_SUPPORTS_VBOOT \
    PRODUCT_SUPPORTS_VERITY \
    PRODUCT_SUPPORTS_VERITY_FEC \
    PRODUCT_OEM_PROPERTIES \
    PRODUCT_SYSTEM_PROPERTY_BLACKLIST \
    PRODUCT_SYSTEM_SERVER_JARS \
    PRODUCT_VBOOT_SIGNING_KEY \
    PRODUCT_VBOOT_SIGNING_SUBKEY \
    PRODUCT_VERITY_SIGNING_KEY \
    PRODUCT_SYSTEM_VERITY_PARTITION \
    PRODUCT_VENDOR_VERITY_PARTITION \
    PRODUCT_DEX_PREOPT_MODULE_CONFIGS \
    PRODUCT_DEX_PREOPT_DEFAULT_FLAGS \
    PRODUCT_DEX_PREOPT_BOOT_FLAGS \
    PRODUCT_SANITIZER_MODULE_CONFIGS \
    PRODUCT_SYSTEM_BASE_FS_PATH \
    PRODUCT_VENDOR_BASE_FS_PATH \
    PRODUCT_SHIPPING_API_LEVEL \


lunch时,比如选择了aosp_arm-eng这个product,那么build system能找到对应的Makefile为build/target/product/aosp_arm.mkaosp_arm.mk(后面将介绍如何找),从而可以获得aosp_arm-eng产品对应的PRODUCT_PACKAGES、PRODUCT_PROPERTY_OVERRIDES等信息,从而可以进行编译


不同的product,有一些公用的信息,如何复用这些公用的信息,是一个问题

比较直观的做法是使用include,但是会带来一些问题:

1、假如A.mk中定义PRODUCT_PACKAGES  := liba.so;B.mk中定义PRODUCT_PACKAGES := libb.so。C.mk想同时编译A.mk和B.mk中指定的模块,在C.mk中先include A,再include B,得到的是PRODUCT_PACKAGES := libb.so,但是我们想要的是PRODUCT_PACKAGES := liba.so libb.so

2、重复include同一个文件的问题


所以android使用了inherit-product的方式,去处理这个问题


二、inherit-product

打开build/target/product/aosp_arm.mk可以看到:

include $(SRC_TARGET_DIR)/product/full.mk

PRODUCT_NAME := aosp_arm


build/target/product/full.mk中为:

$(call inherit-product, $(SRC_TARGET_DIR)/product/aosp_base_telephony.mk)
$(call inherit-product, $(SRC_TARGET_DIR)/board/generic/device.mk)

include $(SRC_TARGET_DIR)/product/emulator.mk

# Overrides
PRODUCT_NAME := full
PRODUCT_DEVICE := generic
PRODUCT_BRAND := Android
PRODUCT_MODEL := AOSP on ARM Emulator

aosp_base_telephony.mk和device.mk包含了一些比较通用的信息


inherit-product的定义为:

#
# $(1): product to inherit
#
# Does three things:
#  1. Inherits all of the variables from $1.
#  2. Records the inheritance in the .INHERITS_FROM variable
#  3. Records that we've visited this node, in ALL_PRODUCTS
#
define inherit-product
  $(if $(findstring ../,$(1)),\
    $(eval np := $(call normalize-paths,$(1))),\
    $(eval np := $(strip $(1))))\
  $(foreach v,$(_product_var_list), \
      $(eval $(v) := $($(v)) $(INHERIT_TAG)$(np))) \
  $(eval inherit_var := \
      PRODUCTS.$(strip $(word 1,$(_include_stack))).INHERITS_FROM) \
  $(eval $(inherit_var) := $(sort $($(inherit_var)) $(np))) \
  $(eval inherit_var:=) \
  $(eval ALL_PRODUCTS := $(sort $(ALL_PRODUCTS) $(word 1,$(_include_stack))))
endef


这几行是用于处理路径问题的,不需要关心

  $(if $(findstring ../,$(1)),\
    $(eval np := $(call normalize-paths,$(1))),\
    $(eval np := $(strip $(1))))\


对于product的每一个信息,记录下信息需要从哪些文件继承

  $(foreach v,$(_product_var_list), \
      $(eval $(v) := $($(v)) $(INHERIT_TAG)$(np))) \

经过上述for循环的处理后,Makefile中包含了这样的代码

PRODUCT_PACKAGES := @inherit:  @inherit:  @inherit:  @inherit:

PRODUCT_PROPERTY_OVERRIDES := @inherit:  @inherit:  @inherit:  @inherit:

......


这里的@inherit只是一个标记,目前还没有起到什么作用。所以inherit-product命令是用来打一些标记,记录继承关系的,在这里并不会执行被继承的.mk文件

这些@inherit的标记是在import-products中进行处理的


后面的几行是定义了一些变量,用于记录一些信息,可以在调试函数中打印出来


三、import-products

我们需要找到android源码中所有product所对应的Makefile,而product对应的Makefile是记录在AndroidProducts.mk文件中的,比如build/target/product/AndroidProducts.mk:

PRODUCT_MAKEFILES := \
    $(LOCAL_DIR)/aosp_arm.mk \
    $(LOCAL_DIR)/full.mk \
    $(LOCAL_DIR)/generic_armv5.mk \
    $(LOCAL_DIR)/emulator_phone.mk \
    $(LOCAL_DIR)/full_x86.mk \
    $(LOCAL_DIR)/aosp_mips.mk \
    $(LOCAL_DIR)/full_mips.mk \
    $(LOCAL_DIR)/aosp_arm64.mk \
    $(LOCAL_DIR)/aosp_mips64.mk \
    $(LOCAL_DIR)/aosp_x86_64.mk

所以,我们需要先找到android源码中所有的AndroidProducts.mk,调用_find-android-products-files函数,即可在device(推荐位置),vendor(旧的位置),product(旧的位置)目录下搜索AndroidProducts.mk,再加上一个默认的$(SRC_TARGET_DIR)/product/AndroidProducts.mk

#
# Functions for including AndroidProducts.mk files
# PRODUCT_MAKEFILES is set up in AndroidProducts.mks.
# Format of PRODUCT_MAKEFILES:
# :
# If the  is the same as the base file name (without dir
# and the .mk suffix) of the product makefile, ":" can be
# omitted.

# Search for AndroidProducts.mks in the given dir.
# $(1): the path to the dir
define _search-android-products-files-in-dir
$(sort $(shell test -d $(1) && find -L $(1) \
  -maxdepth 6 \
  -name .git -prune \
  -o -name AndroidProducts.mk -print))
endef

#
# Returns the list of all AndroidProducts.mk files.
# $(call ) isn't necessary.
#
define _find-android-products-files
$(foreach d, device vendor product,$(call _search-android-products-files-in-dir,$(d))) \
  $(SRC_TARGET_DIR)/product/AndroidProducts.mk
endef


找到所有的AndroidProducts.mk之后,调用get-all-product-makefiles函数,可以得到所有的AndroidProducts.mk中定义的PRODUCT_MAKEFILES,也就可以得到所有product的Makefile

#
# Returns the sorted concatenation of PRODUCT_MAKEFILES
# variables set in the given AndroidProducts.mk files.
# $(1): the list of AndroidProducts.mk files.
#
define get-product-makefiles
$(sort \
  $(foreach f,$(1), \
    $(eval PRODUCT_MAKEFILES :=) \
    $(eval LOCAL_DIR := $(patsubst %/,%,$(dir $(f)))) \
    $(eval include $(f)) \
    $(PRODUCT_MAKEFILES) \
   ) \
  $(eval PRODUCT_MAKEFILES :=) \
  $(eval LOCAL_DIR :=) \
 )
endef

#
# Returns the sorted concatenation of all PRODUCT_MAKEFILES
# variables set in all AndroidProducts.mk files.
# $(call ) isn't necessary.
#
define get-all-product-makefiles
$(call get-product-makefiles,$(_find-android-products-files))
endef


然后调用import-products,导入所有产品的信息

#
# $(1): product makefile list
#
#TODO: check to make sure that products have all the necessary vars defined
define import-products
$(call import-nodes,PRODUCTS,$(1),$(_product_var_list))
endef

import-nodes函数的第一个参数是一个前缀,第二个参数是所有prodcut的Makefile,第三个参数是_product_var_list

#
# $(1): output list variable name, like "PRODUCTS" or "DEVICES"
# $(2): list of makefiles representing nodes to import
# $(3): list of node variable names
#
define import-nodes
$(if \
  $(foreach _in,$(2), \
    $(eval _node_import_context := _nic.$(1).[[$(_in)]]) \
    $(if $(_include_stack),$(eval $(error ASSERTION FAILED: _include_stack \
                should be empty here: $(_include_stack))),) \
    $(eval _include_stack := ) \
    $(call _import-nodes-inner,$(_node_import_context),$(_in),$(3)) \
    $(call move-var-list,$(_node_import_context).$(_in),$(1).$(_in),$(3)) \
    $(eval _node_import_context :=) \
    $(eval $(1) := $($(1)) $(_in)) \
    $(if $(_include_stack),$(eval $(error ASSERTION FAILED: _include_stack \
                should be empty here: $(_include_stack))),) \
   ) \
,)
endef

if是检查,可以先不看


_node_import_context有点像结构体,和一个product对应。

_node_import_context.也有点像结构体,描述该产品的某个.mk(product的主Makefile或者inherit的某个.mk)的信息,如

$(_node_import_context). .PRODUCT_PACKAGES就是某.mk中的PRODUCT_PACKAGES的值

$(_node_import_context)..inherited记录了继承了哪些.mk


在_import-node函数中,会真正得去include某个.mk,得到该.mk中PRODUCT_PACKAGES这样的变量的值,并通过copy-var-list,保存为$(_node_import_context)..PRODUCT_PACKAGES这样的变量。

然后在$(_node_import_context)..inherited中保存继承关系。

由于继承可以有多层,所以会有递归调用_import-node和_import-nodes-inner。

通过_expand-inherited-values函数,使用真实值替换每一层的@inherit:标记。

最终将得到某个product完整的信息,记录在product的主Makefile对应的$(_node_import_context).中。


import-nodes函数最后,通过move-var-list函数,将记录在product的主Makefile对应的$(_node_import_context).中的完整的信息,拷贝为PRODUCTS.. := 格式的数据。

#
# $(1): context prefix
# $(2): makefile representing this node
# $(3): list of node variable names
#
# _include_stack contains the list of included files, with the most recent files first.
define _import-node
  $(eval _include_stack := $(2) $$(_include_stack))
  $(call clear-var-list, $(3))
  $(eval LOCAL_PATH := $(patsubst %/,%,$(dir $(2))))
  $(eval MAKEFILE_LIST :=)
  $(eval include $(2))
  $(eval _included := $(filter-out $(2),$(MAKEFILE_LIST)))
  $(eval MAKEFILE_LIST :=)
  $(eval LOCAL_PATH :=)
  $(call copy-var-list, $(1).$(2), $(3))
  $(call clear-var-list, $(3))

  $(eval $(1).$(2).inherited := \
      $(call get-inherited-nodes,$(1).$(2),$(3)))
  $(call _import-nodes-inner,$(1),$($(1).$(2).inherited),$(3))

  $(call _expand-inherited-values,$(1),$(2),$(3))

  $(eval $(1).$(2).inherited :=)
  $(eval _include_stack := $(wordlist 2,9999,$$(_include_stack)))
endef

#
# This will generate a warning for _included above
#  $(if $(_included), \
#      $(eval $(warning product spec file: $(2)))\
#      $(foreach _inc,$(_included),$(eval $(warning $(space)$(space)$(space)includes: $(_inc)))),)
#

#
# $(1): context prefix
# $(2): list of makefiles representing nodes to import
# $(3): list of node variable names
#
#TODO: Make the "does not exist" message more helpful;
#      should print out the name of the file trying to include it.
define _import-nodes-inner
  $(foreach _in,$(2), \
    $(if $(wildcard $(_in)), \
      $(if $($(1).$(_in).seen), \
        $(eval ### "skipping already-imported $(_in)") \
       , \
        $(eval $(1).$(_in).seen := true) \
        $(call _import-node,$(1),$(strip $(_in)),$(3)) \
       ) \
     , \
      $(error $(1): "$(_in)" does not exist) \
     ) \
   )
endef


如何解析之前的PRODUCT_PACKAGES := @inherit:  @inherit:  @inherit:  @inherit:,得到继承关系?由get-inherited-nodes函数实现

INHERIT_TAG := @inherit:

#
# Walks through the list of variables, each qualified by the prefix,
# and finds instances of words beginning with INHERIT_TAG.  Scrape
# off INHERIT_TAG from each matching word, and return the sorted,
# unique set of those words.
#
# E.g., given
#   PREFIX.A := A $(INHERIT_TAG)aaa B C
#   PREFIX.B := B $(INHERIT_TAG)aaa C $(INHERIT_TAG)bbb D E
# Then
#   $(call get-inherited-nodes,PREFIX,A B)
# returns
#   aaa bbb
#
# $(1): variable prefix
# $(2): list of variables to check
#
define get-inherited-nodes
$(sort \
  $(subst $(INHERIT_TAG),, \
    $(filter $(INHERIT_TAG)%, \
      $(foreach v,$(2),$($(1).$(v))) \
 )))
endef


基于$(_node_import_context)..PRODUCT_PACKAGES这样的变量和$(_node_import_context)..inherited中保存继承关系

PRODUCT_PACKAGES := @inherit:  @inherit:  @inherit:  @inherit:中的@inherit:会被filename1.mk中定义的PRODUCT_PACKAGES所替换,@inherit:会被filename2.mk中定义的PRODUCT_PACKAGES所替换

递归的每一层处理一次,最终将替换所有的@inherit:标记,得到最终的值,记录在product的主Makefile对应的$(_node_import_context).中。

#
# for each variable ( (prefix + name) * vars ):
#   get list of inherited words; if not empty:
#     for each inherit:
#       replace the first occurrence with (prefix + inherited + var)
#       clear the source var so we can't inherit the value twice
#
# $(1): context prefix
# $(2): name of this node
# $(3): list of variable names
#
define _expand-inherited-values
  $(foreach v,$(3), \
    $(eval ### "Shorthand for the name of the target variable") \
    $(eval _eiv_tv := $(1).$(2).$(v)) \
    $(eval ### "Get the list of nodes that this variable inherits") \
    $(eval _eiv_i := \
        $(sort \
            $(patsubst $(INHERIT_TAG)%,%, \
                $(filter $(INHERIT_TAG)%, $($(_eiv_tv)) \
     )))) \
    $(foreach i,$(_eiv_i), \
      $(eval ### "Make sure that this inherit appears only once") \
      $(eval $(_eiv_tv) := \
          $(call uniq-word,$($(_eiv_tv)),$(INHERIT_TAG)$(i))) \
      $(eval ### "Expand the inherit tag") \
      $(eval $(_eiv_tv) := \
          $(strip \
              $(patsubst $(INHERIT_TAG)$(i),$($(1).$(i).$(v)), \
                  $($(_eiv_tv))))) \
      $(eval ### "Clear the child so DAGs don't create duplicate entries" ) \
      $(eval $(1).$(i).$(v) :=) \
      $(eval ### "If we just inherited ourselves, it's a cycle.") \
      $(if $(filter $(INHERIT_TAG)$(2),$($(_eiv_tv))), \
        $(warning Cycle detected between "$(2)" and "$(i)" for context "$(1)") \
        $(error import of "$(2)" failed) \
      ) \
     ) \
   ) \
   $(eval _eiv_tv :=) \
   $(eval _eiv_i :=)
endef


四、lunch

执行source build/envsetup.sh时,会定义lunch函数和add_lunch_combo函数。执行lunch时,可以列出所有的产品,可以进行选择。add_lunch_combo是用来添加新的产品的,否则是无法在lunch时看到的。

在build/envsetup.sh的最后,有如下代码,搜索每一款产品所对应的vendorsetup.sh并执行,vendorsetup.sh中就包含了add_lunch_combo的调用,将本产品添加到lunch菜单中

# Execute the contents of any vendorsetup.sh files we can find.
for f in `test -d device && find -L device -maxdepth 4 -name 'vendorsetup.sh' 2> /dev/null | sort` \
         `test -d vendor && find -L vendor -maxdepth 4 -name 'vendorsetup.sh' 2> /dev/null | sort` \
         `test -d product && find -L product -maxdepth 4 -name 'vendorsetup.sh' 2> /dev/null | sort`
do
    echo "including $f"
    . $f
done


当我们lunch aosp_arm-eng时,lunch函数中会分隔得到TARGET_PRODUCT=aosp_arm,TARGET_BUILD_VARIANT=eng


五、使用product特定的信息

以PRODUCT_PACKAGES为例,如何得知aosp_arm-eng这个产品,需要编译哪些模块呢?


build/core/main.mk文件中的product_MODULES是需要编译的所有的模块,通过如下代码赋值

product_MODULES := $(PRODUCTS.$(INTERNAL_PRODUCT).PRODUCT_PACKAGES)

INTERNAL_PRODUCT应当是build/target/product/aosp_arm.mk,从而我们可以使用之前得到的PRODUCTS.build/target/product/aosp_arm.mk.PRODUCT_PACKAGES变量

但是如何得知INTERNAL_PRODUCT是build/target/product/aosp_arm.mk呢,通过遍历所以产品的Makefile,找到PRODUCT_NAME和TARGET_PRODUCT相等的那个产品的Makefile即可

INTERNAL_PRODUCT := $(call resolve-short-product-name, $(TARGET_PRODUCT))

#
# Returns the product makefile path for the product with the provided name
#
# $(1): short product name like "generic"
#
define _resolve-short-product-name
  $(eval pn := $(strip $(1)))
  $(eval p := \
      $(foreach p,$(PRODUCTS), \
          $(if $(filter $(pn),$(PRODUCTS.$(p).PRODUCT_NAME)), \
            $(p) \
       )) \
   )
  $(eval p := $(sort $(p)))
  $(if $(filter 1,$(words $(p))), \
    $(p), \
    $(if $(filter 0,$(words $(p))), \
      $(error No matches for product "$(pn)"), \
      $(error Product "$(pn)" ambiguous: matches $(p)) \
    ) \
  )
endef
define resolve-short-product-name
$(strip $(call _resolve-short-product-name,$(1)))
endef


这样,android build system就完成了product的继承,所有product的载入,以及根据lunch的product,找到需要编译的product的信息


参考:

http://baohaojun.github.io/blog/2012/09/17/Android-Product-Makefiles.html





你可能感兴趣的