Airtest Project 包含了两个框架，一个是 Airtest 一个是 Poco，这两个框架都是Python 的第三方库。

Airtest 是一个跨平台的、 基于图像识别 的UI自动化测试框架，适用于游戏和App，支持平台有Windows、Android和iOS.

Poco 是一款 基于UI控件识别 的自动化测试框架，目前支持Android原生、iOS原生、Unity3D、cocos2dx、UE4和Egret等平台，也可以在其他引擎中自行接入poco-sdk来使用。

                                                                                                                ——官方文档

无论使用哪种方式，要远程控制手机，先把手机上的adb端口打开。先用USB线把手机连上电脑。 在终端里面执行命令：adb tcpip 5555其中的端口号5555你可以自行设定为其他的端口号，但不能和已有的端口冲突。

adb connect 手机IP:端口

有个device参数，只要输入设备的URI，会自动帮我们连接设备。

from airtest.core.api import *
# 连接第一台手机
dev1 = connect_device("Android://127.0.0.1:5037/serial1") 
# 连接第二台手机
dev2 = connect_device("Android://127.0.0.1:5037/serial2") # IP和接口默认情况下不会变，所以可以简写为
dev1 = connect_device("Android:///serial1") 
dev2 = connect_device("Android:///serial2")

同一时间，只有一台设备是激活状态可操作的。所以操作完一台后，要切换到第2台去操作

airtest部分源码解析

# settings.py# -*- coding: utf-8 -*-
from airtest.utils.resolution import cocos_min_strategy
import os
import cv2
from distutils.version import LooseVersionclass Settings(object):DEBUG = FalseLOG_DIR = NoneLOG_FILE = "log.txt"RESIZE_METHOD = staticmethod(cocos_min_strategy)# keypoint matching: kaze/brisk/akaze/orb, contrib: sift/surf/briefCVSTRATEGY = ["mstpl", "tpl", "surf", "brisk"]  # 匹配图片的算法的列表if LooseVersion(cv2.__version__) > LooseVersion('3.4.2'):CVSTRATEGY = ["mstpl", "tpl", "sift", "brisk"]KEYPOINT_MATCHING_PREDICTION = TrueTHRESHOLD = 0.7  # [0, 1]THRESHOLD_STRICT = None  # dedicated parameter for assert_existsOPDELAY = 0.1FIND_TIMEOUT = 20   # 超时的默认值FIND_TIMEOUT_TMP = 3PROJECT_ROOT = os.environ.get("PROJECT_ROOT", "")  # for ``using`` other scriptSNAPSHOT_QUALITY = 10  # 1-100 https://pillow.readthedocs.io/en/5.1.x/handbook/image-file-formats.html#jpeg# Image compression size, e.g. 1200, means that the size of the screenshot does not exceed 1200*1200IMAGE_MAXSIZE = os.environ.get("IMAGE_MAXSIZE", None)SAVE_IMAGE = True

#cv.py#!/usr/bin/env python
# -*- coding: utf-8 -*-""""Airtest图像识别专用."""import os
import sys
import time
import types
from six import PY3
from copy import deepcopyfrom airtest import aircv
from airtest.aircv import cv2
from airtest.core.helper import G, logwrap
from airtest.core.settings import Settings as ST  # noqa
from airtest.core.error import TargetNotFoundError, InvalidMatchingMethodError
from airtest.utils.transform import TargetPosfrom airtest.aircv.template_matching import TemplateMatching
from airtest.aircv.multiscale_template_matching import MultiScaleTemplateMatching,MultiScaleTemplateMatchingPre
from airtest.aircv.keypoint_matching import KAZEMatching, BRISKMatching, AKAZEMatching, ORBMatching
from airtest.aircv.keypoint_matching_contrib import SIFTMatching, SURFMatching, BRIEFMatchingMATCHING_METHODS = {"tpl": TemplateMatching,"mstpl": MultiScaleTemplateMatchingPre,"gmstpl": MultiScaleTemplateMatching,"kaze": KAZEMatching,"brisk": BRISKMatching,"akaze": AKAZEMatching,"orb": ORBMatching,"sift": SIFTMatching,"surf": SURFMatching,"brief": BRIEFMatching,
}@logwrap
def loop_find(query, timeout=ST.FIND_TIMEOUT, threshold=None, interval=0.5, intervalfunc=None):"""Search for image template in the screen until timeoutArgs:query: image template to be found in screenshottimeout: time interval how long to look for the image templatethreshold: default is Noneinterval: sleep interval before next attempt to find the image templateintervalfunc: function that is executed after unsuccessful attempt to find the image templateRaises:TargetNotFoundError: when image template is not found in screenshotReturns:TargetNotFoundError if image template not found, otherwise returns the position where the image template hasbeen found in screenshot# https://mp.weixin.qq.com/s?__biz=MzA5NDYxNTU5NA==&mid=2247491438&idx=1&sn=d69054bcbe3a20ed412309c6057fd4be&chksm=904abe2ba73d373d639222960517e085bedaaacb96364ca7e2dcc97bf75d3d62bafbe0d8373c&scene=21#wechat_redirect
在屏幕中搜索图像模板，直到超时
参数：
查询：要在屏幕截图中找到的图像模板
timeout：查找图像模板的时间间隔
阈值：默认值为“None”,识别阈值，浮点类型，范围是[0.0, 1.0]，默认0.7。也就是,当识别可信度=>0.7时就认为是匹配的。对于计算机来说，不存在2张完全一样的图片，计算机只能告诉你2张图片的相似程度。比如相似度是0.9(90%)就是比较像，相似度是0.5(50%)就是不太像。计算机只会告诉你相似度，那这2张图算不算匹配，是由人通过阈值决定的。比如我们说只要相似度70%以上，就算是一样的。可以指定某个图片的阈值，如touch(Template(r"tpl1556019871196.png", threshold=0.9))假如我们想让所有图片的阈值都是0.9，可以通过全局设置：airtest.core.api中包含了一个名为ST的变量，即为全局设置ST.THRESHOLD = 0.9  # 设置全局的匹配阈值为0.9# 未写明图片threshold，使用上面全局的ST.THRESHOLD=0.9touch(Template(r"1.png", record_pos=(0.79, 0.32), resolution=(107, 1164)))interval：下次尝试查找图像模板之前的睡眠时间间隔
intervalfunc：在尝试查找图像模板失败后执行的函数
增加：
TargetNotFoundError:在屏幕截图中找不到图像模板时
Returns：
如果未找到图像模板，则返回TargetNotFoundError，否则返回屏幕截图中找到图像模板的位置"""# loop_find整体逻辑就是循环去屏幕截图上找图，找到返回其坐标，超时未找到报错。# loop_find的第1个参数query就是前面传入的Template类实例(截的图)G.LOGGING.info("Try finding: %s", query)start_time = time.time()while True:screen = G.DEVICE.snapshot(filename=None, quality=ST.SNAPSHOT_QUALITY)if screen is None:G.LOGGING.warning("Screen is None, may be locked")else:if threshold:query.threshold = thresholdmatch_pos = query.match_in(screen)  #给手机截图赋值给screen，然后在截图中查找给定图片，用的方法是Template类中的match_in方法。if match_pos:try_log_screen(screen)return match_posif intervalfunc is not None:intervalfunc()# 超时则raise，未超时则进行下次循环:if (time.time() - start_time) > timeout:try_log_screen(screen)raise TargetNotFoundError('Picture %s not found in screen' % query)else:time.sleep(interval)@logwrap
def try_log_screen(screen=None, quality=None, max_size=None):"""Save screenshot to fileArgs:screen: screenshot to be savedquality: The image quality, default is ST.SNAPSHOT_QUALITYmax_size: the maximum size of the picture, e.g 1200Returns:{"screen": filename, "resolution": aircv.get_resolution(screen)}将屏幕截图保存到文件
参数：
屏幕：要保存的屏幕截图
quality:图像质量，默认为ST.SNAPSHOT_quality
max_size：图片的最大大小，例如1200
Returns：
｛“屏幕”：文件名，“分辨率”：aircv.get_resolution（屏幕）｝"""if not ST.LOG_DIR or not ST.SAVE_IMAGE:returnif not quality:quality = ST.SNAPSHOT_QUALITYif not max_size:max_size = ST.IMAGE_MAXSIZEif screen is None:screen = G.DEVICE.snapshot(quality=quality)filename = "%(time)d.jpg" % {'time': time.time() * 1000}filepath = os.path.join(ST.LOG_DIR, filename)if screen is not None:aircv.imwrite(filepath, screen, quality, max_size=max_size)return {"screen": filename, "resolution": aircv.get_resolution(screen)}return Noneclass Template(object):"""picture as touch/swipe/wait/exists target and extra info for cv match图片作为触摸/滑动/等待/存在目标和简历匹配的额外信息filename: pic filename文件名：pic文件名(文件路径)target_pos: ret which pos in the pictarget_pos:ret图片中的位置touch(Template(r"1.png", target_pos=8))record_pos: pos in screen when recordingrecord_pos:录制时屏幕中的位置.图片坐标对应手机屏幕中心点的偏移值相对于手机分辨率的百分比，匹配时会优先匹配这附近的画面。这样Airtest在图像匹配时，会优化在这个坐标区域附件查找，提高查找图片的速度和精确度。如果在这个区域找不到，才会将查找范围扩大至整个画面。resolution: screen resolution when recording分辨率：录制时的屏幕分辨率.手机分辨率。当脚本执行时的手机不是录制时的手机时，Airtest会对屏幕截图按照分辨率进行缩放，最大程度兼容跨分辨率匹配。但如果不同手机的分辨率比例相差大，仍会导致匹配不到图片，所以可能同一张图，不同手机需要准备2张或以上的图片做为查找图片。rgb: 识别结果是否使用rgb三通道进行校验.是否开启彩色识别，Bool类型，默认False.rgb=False时，Airtest会先将图像转为灰度图再进行识别；为True时，指定使用彩色图像进行识别。一般情况下，我们都用默认的False即可，但假如画面上有多个形状相同颜色不同图片时，就要设为True。scale_max: 多尺度模板匹配最大范围.Airtest1.2.0新增图像识别算法mstpl专用参数，用于调节匹配的最大范围默认值 800， 推荐值 740, 800, 1000 取值范围 [700 , 2000]。比如截屏为(500,2000)，scale_max=1000，则在匹配前截屏会resize为(250,1000)。resize后图片大小少了一倍，理论上匹配速度也会变快，但因为缩小后，更不容易匹配较小的UI，所以如果要查找的目标UI很小的话，可以适当增大这个数值。scale_step: 多尺度模板匹配搜索步长.Airtest1.2.0新增图像识别算法mstpl专用参数，用于控制搜索比例步长，在匹配时，会以截图最长边*scale_step的步长进行搜索，默认值0.005，推荐值 0.02, 0.005, 0.001 取值范围 [0.001, 0.1]。如果要查找的目标UI很小的话，可以适当减小这个数值(通常不需要调整该值，但如果有跨分辨率匹配失败，可以尝试减少，相应的匹配时间则会大大增加)"""def __init__(self, filename, threshold=None, target_pos=TargetPos.MID, record_pos=None, resolution=(), rgb=False, scale_max=800, scale_step=0.005):self.filename = filenameself._filepath = Noneself.threshold = threshold or ST.THRESHOLDself.target_pos = target_posself.record_pos = record_posself.resolution = resolutionself.rgb = rgbself.scale_max = scale_maxself.scale_step = scale_step@propertydef filepath(self):if self._filepath:return self._filepathfor dirname in G.BASEDIR:filepath = os.path.join(dirname, self.filename)if os.path.isfile(filepath):self._filepath = filepathreturn self._filepathreturn self.filenamedef __repr__(self):filepath = self.filepath if PY3 else self.filepath.encode(sys.getfilesystemencoding())return "Template(%s)" % filepathdef match_in(self, screen):match_result = self._cv_match(screen)   # 图像匹配G.LOGGING.debug("match result: %s", match_result)if not match_result:return Nonefocus_pos = TargetPos().getXY(match_result, self.target_pos)    # 如果找到，会返回9宫点中要求的坐标return focus_posdef match_all_in(self, screen):image = self._imread()image = self._resize_image(image, screen, ST.RESIZE_METHOD)return self._find_all_template(image, screen)@logwrapdef _cv_match(self, screen):# in case image file not exist in current directory:    #如果当前目录中不存在图像文件：ori_image = self._imread()  #读取图像image = self._resize_image(ori_image, screen, ST.RESIZE_METHOD) # 根据分辨率，将输入的截图适配成 等待模板匹配的截图.之后会循环各种算法去匹配图片.传入的图像需要进行缩放变化，写用例时候的截图进行变换后转换成跑用例时候的截图，以提高匹配成功率ret = Nonefor method in ST.CVSTRATEGY:    # 算法去匹配图片ST.CVSTRATEGY# get function definition and execute:func = MATCHING_METHODS.get(method, None)if func is None:raise InvalidMatchingMethodError("Undefined method in CVSTRATEGY: '%s', try 'kaze'/'brisk'/'akaze'/'orb'/'surf'/'sift'/'brief' instead." % method)else:if method in ["mstpl", "gmstpl"]:ret = self._try_match(func, ori_image, screen, threshold=self.threshold, rgb=self.rgb, record_pos=self.record_pos,resolution=self.resolution, scale_max=self.scale_max, scale_step=self.scale_step)else:ret = self._try_match(func, image, screen, threshold=self.threshold, rgb=self.rgb)if ret:breakreturn ret@staticmethoddef _try_match(func, *args, **kwargs):      # 匹配方法为_try_matchG.LOGGING.debug("try match with %s" % func.__name__)try:ret = func(*args, **kwargs).find_best_result()  # 不同的算法对应不同的find_best_result()方法，目前一共有4种except aircv.NoModuleError as err:G.LOGGING.warning("'surf'/'sift'/'brief' is in opencv-contrib module. You can use 'tpl'/'kaze'/'brisk'/'akaze'/'orb' in CVSTRATEGY, or reinstall opencv with the contrib module.")return Noneexcept aircv.BaseError as err:G.LOGGING.debug(repr(err))return Noneelse:return retdef _imread(self):return aircv.imread(self.filepath)def _find_all_template(self, image, screen):    # find_all_template是返回所有大于指定置信度的结果。return TemplateMatching(image, screen, threshold=self.threshold, rgb=self.rgb).find_all_results()def _find_keypoint_result_in_predict_area(self, func, image, screen):if not self.record_pos:return None# calc predict area in screenimage_wh, screen_resolution = aircv.get_resolution(image), aircv.get_resolution(screen)xmin, ymin, xmax, ymax = Predictor.get_predict_area(self.record_pos, image_wh, self.resolution,screen_resolution)# crop predict image from screenpredict_area = aircv.crop_image(screen, (xmin, ymin, xmax, ymax))if not predict_area.any():return None# keypoint matching in predicted area:ret_in_area = func(image, predict_area, threshold=self.threshold, rgb=self.rgb)# calc cv ret if foundif not ret_in_area:return Noneret = deepcopy(ret_in_area)if "rectangle" in ret:for idx, item in enumerate(ret["rectangle"]):ret["rectangle"][idx] = (item[0] + xmin, item[1] + ymin)ret["result"] = (ret_in_area["result"][0] + xmin, ret_in_area["result"][1] + ymin)

#template_matching.py# !/usr/bin/env python
# -*- coding: utf-8 -*-"""模板匹配.对用户提供的调节参数:1. threshod: 筛选阈值，默认为0.82. rgb: 彩色三通道,进行彩色权识别.
"""import cv2
import timefrom airtest.utils.logger import get_logger
from .utils import generate_result, check_source_larger_than_search, img_mat_rgb_2_gray, print_run_time
from .cal_confidence import cal_rgb_confidenceLOGGING = get_logger(__name__)class TemplateMatching(object):"""模板匹配."""METHOD_NAME = "Template"MAX_RESULT_COUNT = 10def __init__(self, im_search, im_source, threshold=0.8, rgb=True):super(TemplateMatching, self).__init__()self.im_source = im_sourceself.im_search = im_searchself.threshold = thresholdself.rgb = rgb@print_run_timedef find_all_results(self):"""基于模板匹配查找多个目标区域的方法."""# 第一步：校验图像输入check_source_larger_than_search(self.im_source, self.im_search)# 第二步：计算模板匹配的结果矩阵resres = self._get_template_result_matrix()# 第三步：依次获取匹配结果result = []h, w = self.im_search.shape[:2]while True:# 本次循环中,取出当前结果矩阵中的最优值min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res)# 求取可信度:confidence = self._get_confidence_from_matrix(max_loc, max_val, w, h)if confidence < self.threshold or len(result) > self.MAX_RESULT_COUNT:break# 求取识别位置: 目标中心 + 目标区域:middle_point, rectangle = self._get_target_rectangle(max_loc, w, h)one_good_match = generate_result(middle_point, rectangle, confidence)result.append(one_good_match)# 屏蔽已经取出的最优结果,进入下轮循环继续寻找:# cv2.floodFill(res, None, max_loc, (-1000,), max(max_val, 0), flags=cv2.FLOODFILL_FIXED_RANGE)cv2.rectangle(res, (int(max_loc[0] - w / 2), int(max_loc[1] - h / 2)), (int(max_loc[0] + w / 2), int(max_loc[1] + h / 2)), (0, 0, 0), -1)return result if result else None@print_run_timedef find_best_result(self):"""基于kaze进行图像识别，只筛选出最优区域.""""""函数功能：找到最优结果."""# 第一步：校验图像输入check_source_larger_than_search(self.im_source, self.im_search)# 第二步：计算模板匹配的结果矩阵resres = self._get_template_result_matrix()# 第三步：依次获取匹配结果min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(res) # min_val 最小值,max_val 最大值,min_loc 最小值位置（区域的左上角坐标）,max_loc 最大值位置（区域的左上角坐标）h, w = self.im_search.shape[:2]# 求取可信度:confidence = self._get_confidence_from_matrix(max_loc, max_val, w, h)# 求取识别位置: 目标中心 + 目标区域:middle_point, rectangle = self._get_target_rectangle(max_loc, w, h)best_match = generate_result(middle_point, rectangle, confidence)LOGGING.debug("[%s] threshold=%s, result=%s" % (self.METHOD_NAME, self.threshold, best_match))return best_match if confidence >= self.threshold else Nonedef _get_confidence_from_matrix(self, max_loc, max_val, w, h):"""根据结果矩阵求出confidence."""# 求取可信度:if self.rgb:# 如果有颜色校验,对目标区域进行BGR三通道校验:img_crop = self.im_source[max_loc[1]:max_loc[1] + h, max_loc[0]: max_loc[0] + w]confidence = cal_rgb_confidence(img_crop, self.im_search)else:confidence = max_valreturn confidencedef _get_template_result_matrix(self):"""求取模板匹配的结果矩阵."""# cv2.matchTemplate是opencv的方法，它的返回值是个矩阵，相当于用小图在大图上滑动，从左上角开始，每次移动一个像素，然后计算一个匹配结果，最终形成结果矩阵。# 灰度识别: cv2.matchTemplate( )只能处理灰度图片参数s_gray, i_gray = img_mat_rgb_2_gray(self.im_search), img_mat_rgb_2_gray(self.im_source)return cv2.matchTemplate(i_gray, s_gray, cv2.TM_CCOEFF_NORMED)  # 匹配方法:TM_CCOEFF_NORMED计算归一化相关系数，计算出来的值越接近1越相关def _get_target_rectangle(self, left_top_pos, w, h):"""根据左上角点和宽高求出目标区域."""x_min, y_min = left_top_pos# 中心位置的坐标:x_middle, y_middle = int(x_min + w / 2), int(y_min + h / 2)# 左下(min,max)->右下(max,max)->右上(max,min)left_bottom_pos, right_bottom_pos = (x_min, y_min + h), (x_min + w, y_min + h)right_top_pos = (x_min + w, y_min)# 点击位置:middle_point = (x_middle, y_middle)# 识别目标区域: 点序:左上->左下->右下->右上, 左上(min,min)右下(max,max)rectangle = (left_top_pos, left_bottom_pos, right_bottom_pos, right_top_pos)return middle_point, rectangle

# api.py# -*- coding: utf-8 -*-
"""
This module contains the Airtest Core APIs.
"""
import os
import timefrom six.moves.urllib.parse import parse_qsl, urlparsefrom airtest.core.cv import Template, loop_find, try_log_screen
from airtest.core.error import TargetNotFoundError
from airtest.core.settings import Settings as ST
from airtest.utils.compat import script_log_dir
from airtest.core.helper import (G, delay_after_operation, import_device_cls,logwrap, set_logdir, using, log)
# Assertions
from airtest.core.assertions import (assert_exists, assert_not_exists, assert_equal, assert_not_equal,  # noqaassert_true, assert_false, assert_is, assert_is_not,assert_is_none, assert_is_not_none, assert_in, assert_not_in,assert_is_instance, assert_not_is_instance)"""
Device Setup APIs
"""def init_device(platform="Android", uuid=None, **kwargs):"""Initialize device if not yet, and set as current device.:param platform: Android, IOS or Windows:param uuid: uuid for target device, e.g. serialno for Android, handle for Windows, uuid for iOS:param kwargs: Optional platform specific keyword args, e.g. `cap_method=JAVACAP` for Android:return: device instance:Example:>>> init_device(platform="Android",uuid="SJE5T17B17", cap_method="JAVACAP")>>> init_device(platform="Windows",uuid="123456")"""cls = import_device_cls(platform)dev = cls(uuid, **kwargs)# Add device instance in G and set as current device.G.add_device(dev)return devdef connect_device(uri):"""Initialize device with uri, and set as current device.:param uri: an URI where to connect to device, e.g. `android://adbhost:adbport/serialno?param=value¶m2=value2`:return: device instance:Example:>>> connect_device("Android:///")  # local adb device using default params>>> # local device with serial number SJE5T17B17 and custom params>>> connect_device("Android:///SJE5T17B17?cap_method=javacap&touch_method=adb")>>> # remote device using custom params Android://adbhost:adbport/serialno>>> connect_device("Android://127.0.0.1:5037/10.254.60.1:5555")>>> connect_device("Windows:///")  # connect to the desktop>>> connect_device("Windows:///123456")  # Connect to the window with handle 123456>>> connect_device("Windows:///123456?foreground=False")  # Connect to the window without setting it foreground>>> connect_device("iOS:///127.0.0.1:8100")  # iOS device>>> connect_device("iOS:///http://localhost:8100/?mjpeg_port=9100")  # iOS with mjpeg port"""d = urlparse(uri)platform = d.schemehost = d.netlocuuid = d.path.lstrip("/")params = dict(parse_qsl(d.query))if host:params["host"] = host.split(":")dev = init_device(platform, uuid, **params)return devdef device():"""Return the current active device.:return: current device instance:Example:>>> dev = device()>>> dev.touch((100, 100))"""return G.DEVICEdef set_current(idx):"""Set current active device.:param idx: uuid or index of initialized device instance:raise IndexError: raised when device idx is not found:return: None:platforms: Android, iOS, Windows:Example:>>> # switch to the first phone currently connected>>> set_current(0)>>> # switch to the phone with serial number serialno1>>> set_current("serialno1")"""dev_dict = {dev.uuid: dev for dev in G.DEVICE_LIST}if idx in dev_dict:current_dev = dev_dict[idx]elif isinstance(idx, int) and idx < len(G.DEVICE_LIST):current_dev = G.DEVICE_LIST[idx]else:raise IndexError("device idx not found in: %s or %s" % (list(dev_dict.keys()), list(range(len(G.DEVICE_LIST)))))G.DEVICE = current_devdef auto_setup(basedir=None, devices=None, logdir=None, project_root=None, compress=None):"""Auto setup running env and try connect android device if not device connected.:param basedir: basedir of script, __file__ is also acceptable.:param devices: connect_device uri in list.:param logdir: log dir for script report, default is None for no log, set to ``True`` for ``/log``.:param project_root: project root dir for `using` api.:param compress: The compression rate of the screenshot image, integer in range [1, 99], default is 10:Example:>>> auto_setup(__file__)>>> auto_setup(__file__, devices=["Android://127.0.0.1:5037/SJE5T17B17"],...             logdir=True, project_root=r"D:\\test\\logs", compress=90)"""if basedir:if os.path.isfile(basedir):basedir = os.path.dirname(basedir)if basedir not in G.BASEDIR:G.BASEDIR.append(basedir)if devices:for dev in devices:connect_device(dev)if logdir:logdir = script_log_dir(basedir, logdir)set_logdir(logdir)if project_root:ST.PROJECT_ROOT = project_rootif compress:ST.SNAPSHOT_QUALITY = compress"""
Device Operations
"""@logwrap
def shell(cmd):"""Start remote shell in the target device and execute the command:param cmd: command to be run on device, e.g. "ls /data/local/tmp":return: the output of the shell cmd:platforms: Android:Example:>>> # Execute commands on the current device adb shell ls>>> print(shell("ls"))>>> # Execute adb instructions for specific devices>>> dev = connect_device("Android:///device1")>>> dev.shell("ls")>>> # Switch to a device and execute the adb command>>> set_current(0)>>> shell("ls")"""return G.DEVICE.shell(cmd)@logwrap
def start_app(package, activity=None):"""Start the target application on device:param package: name of the package to be started, e.g. "com.netease.my":param activity: the activity to start, default is None which means the main activity:return: None:platforms: Android, iOS:Example:>>> start_app("com.netease.cloudmusic")>>> start_app("com.apple.mobilesafari")  # on iOS"""G.DEVICE.start_app(package, activity)@logwrap
def stop_app(package):"""Stop the target application on device:param package: name of the package to stop, see also `start_app`:return: None:platforms: Android, iOS:Example:>>> stop_app("com.netease.cloudmusic")"""G.DEVICE.stop_app(package)@logwrap
def clear_app(package):"""Clear data of the target application on device:param package: name of the package,  see also `start_app`:return: None:platforms: Android:Example:>>> clear_app("com.netease.cloudmusic")"""G.DEVICE.clear_app(package)@logwrap
def install(filepath, **kwargs):"""Install application on device:param filepath: the path to file to be installed on target device:param kwargs: platform specific `kwargs`, please refer to corresponding docs:return: None:platforms: Android:Example:>>> install(r"D:\\demo\\test.apk")>>> # adb install -r -t D:\\demo\\test.apk>>> install(r"D:\\demo\\test.apk", install_options=["-r", "-t"])"""return G.DEVICE.install_app(filepath, **kwargs)@logwrap
def uninstall(package):"""Uninstall application on device:param package: name of the package, see also `start_app`:return: None:platforms: Android:Example:>>> uninstall("com.netease.cloudmusic")"""return G.DEVICE.uninstall_app(package)@logwrap
def snapshot(filename=None, msg="", quality=None, max_size=None):"""Take the screenshot of the target device and save it to the file.:param filename: name of the file where to save the screenshot. If the relative path is provided, the defaultlocation is ``ST.LOG_DIR``:param msg: short description for screenshot, it will be recorded in the report:param quality: The image quality, integer in range [1, 99], default is 10:param max_size: the maximum size of the picture, e.g 1200:return: {"screen": filename, "resolution": resolution of the screen} or None:platforms: Android, iOS, Windows:Example:>>> snapshot(msg="index")>>> # save the screenshot to test.jpg>>> snapshot(filename="test.png", msg="test")The quality and size of the screenshot can be set::>>> # Set the screenshot quality to 30>>> ST.SNAPSHOT_QUALITY = 30>>> # Set the screenshot size not to exceed 600*600>>> # if not set, the default size is the original image size>>> ST.IMAGE_MAXSIZE = 600>>> # The quality of the screenshot is 30, and the size does not exceed 600*600>>> touch((100, 100))>>> # The quality of the screenshot of this sentence is 90>>> snapshot(filename="test.png", msg="test", quality=90)>>> # The quality of the screenshot is 90, and the size does not exceed 1200*1200>>> snapshot(filename="test2.png", msg="test", quality=90, max_size=1200)"""if not quality:quality = ST.SNAPSHOT_QUALITYif not max_size and ST.IMAGE_MAXSIZE:max_size = ST.IMAGE_MAXSIZEif filename:if not os.path.isabs(filename):logdir = ST.LOG_DIR or "."filename = os.path.join(logdir, filename)screen = G.DEVICE.snapshot(filename, quality=quality, max_size=max_size)return try_log_screen(screen, quality=quality, max_size=max_size)else:return try_log_screen(quality=quality, max_size=max_size)@logwrap
def wake():"""Wake up and unlock the target device:return: None:platforms: Android:Example:>>> wake().. note:: Might not work on some models"""G.DEVICE.wake()@logwrap
def home():"""Return to the home screen of the target device.:return: None:platforms: Android, iOS:Example:>>> home()"""G.DEVICE.home()@logwrap
def touch(v, times=1, **kwargs):"""在设备屏幕上执行触摸操作:param(参数) v: 要触摸的目标，“Template”(图片)实例或绝对坐标（x，y）:param times: 要执行的触摸次数:param kwargs: 特定于平台的“kwargs”，请参阅相应的文档:return(返回): 要单击的最终位置，例如（100，100）:platforms(平台): Android, Windows, iOS:Example:单击绝对坐标::>>> touch((100, 100))单击图片的中心（模板对象）::>>> touch(Template(r"tpl1606730579419.png", target_pos=5))单击2次::>>> touch((100, 100), times=2)在Android和Windows平台下，您可以设置点击持续时间duration::>>> touch((100, 100), duration=2)右键单击（Windows）::>>> touch((100, 100), right_click=True)"""if isinstance(v, Template):pos = loop_find(v, timeout=ST.FIND_TIMEOUT) # 通过loop_find去循环找图,超时时间ST.FIND_TIMEOUT默认是20Selse:try_log_screen()pos = vfor _ in range(times):G.DEVICE.touch(pos, **kwargs)time.sleep(0.05)delay_after_operation()return posclick = touch  # click is alias of touch# 通过loop_find去循环找图，找到图片的话会返回坐标，后面的代码会去点击这个坐标，就完成了touch操作。@logwrap
def double_click(v):"""Perform double click:param v: target to touch, either a ``Template`` instance or absolute coordinates (x, y):return: finial position to be clicked:Example:>>> double_click((100, 100))>>> double_click(Template(r"tpl1606730579419.png"))"""if isinstance(v, Template):pos = loop_find(v, timeout=ST.FIND_TIMEOUT)else:try_log_screen()pos = vG.DEVICE.double_click(pos)delay_after_operation()return pos@logwrap
def swipe(v1, v2=None, vector=None, **kwargs):"""Perform the swipe action on the device screen.There are two ways of assigning the parameters* ``swipe(v1, v2=Template(...))``   # swipe from v1 to v2* ``swipe(v1, vector=(x, y))``      # swipe starts at v1 and moves along the vector.:param v1: the start point of swipe,either a Template instance or absolute coordinates (x, y):param v2: the end point of swipe,either a Template instance or absolute coordinates (x, y):param vector: a vector coordinates of swipe action, either absolute coordinates (x, y) or percentage ofscreen e.g.(0.5, 0.5):param **kwargs: platform specific `kwargs`, please refer to corresponding docs:raise Exception: general exception when not enough parameters to perform swap action have been provided:return: Origin position and target position:platforms: Android, Windows, iOS:Example:>>> swipe(Template(r"tpl1606814865574.png"), vector=[-0.0316, -0.3311])>>> swipe((100, 100), (200, 200))Custom swiping duration and number of steps(Android and iOS)::>>> # swiping lasts for 1 second, divided into 6 steps>>> swipe((100, 100), (200, 200), duration=1, steps=6)"""if isinstance(v1, Template):try:pos1 = loop_find(v1, timeout=ST.FIND_TIMEOUT)except TargetNotFoundError:# 如果由图1滑向图2，图1找不到，会导致图2的文件路径未被初始化，可能在报告中不能正确显示if v2 and isinstance(v2, Template):v2.filepathraiseelse:try_log_screen()pos1 = v1if v2:if isinstance(v2, Template):pos2 = loop_find(v2, timeout=ST.FIND_TIMEOUT_TMP)else:pos2 = v2elif vector:if vector[0] <= 1 and vector[1] <= 1:w, h = G.DEVICE.get_current_resolution()vector = (int(vector[0] * w), int(vector[1] * h))pos2 = (pos1[0] + vector[0], pos1[1] + vector[1])else:raise Exception("no enough params for swipe")G.DEVICE.swipe(pos1, pos2, **kwargs)delay_after_operation()return pos1, pos2@logwrap
def pinch(in_or_out='in', center=None, percent=0.5):"""Perform the pinch action on the device screen:param in_or_out: pinch in or pinch out, enum in ["in", "out"]:param center: center of pinch action, default as None which is the center of the screen:param percent: percentage of the screen of pinch action, default is 0.5:return: None:platforms: Android:Example:Pinch in the center of the screen with two fingers::>>> pinch()Take (100,100) as the center and slide out with two fingers::>>> pinch('out', center=(100, 100))"""try_log_screen()G.DEVICE.pinch(in_or_out=in_or_out, center=center, percent=percent)delay_after_operation()@logwrap
def keyevent(keyname, **kwargs):"""Perform key event on the device:param keyname: platform specific key name:param **kwargs: platform specific `kwargs`, please refer to corresponding docs:return: None:platforms: Android, Windows, iOS:Example:* ``Android``: it is equivalent to executing ``adb shell input keyevent KEYNAME`` ::>>> keyevent("HOME")>>> # The constant corresponding to the home key is 3>>> keyevent("3")  # same as keyevent("HOME")>>> keyevent("BACK")>>> keyevent("KEYCODE_DEL").. seealso::Module :py:mod:`airtest.core.android.adb.ADB.keyevent`Equivalent to calling the ``android.adb.keyevent()```Android Keyevent `_Documentation for more ``Android.KeyEvent``* ``Windows``: Use ``pywinauto.keyboard`` module for key input::>>> keyevent("{DEL}")>>> keyevent("%{F4}")  # close an active window with Alt+F4.. seealso::Module :py:mod:`airtest.core.win.win.Windows.keyevent``pywinauto.keyboard `_Documentation for ``pywinauto.keyboard``* ``iOS``: Only supports home/volumeUp/volumeDown::>>> keyevent("HOME")>>> keyevent("volumeUp")"""G.DEVICE.keyevent(keyname, **kwargs)delay_after_operation()@logwrap
def text(text, enter=True, **kwargs):"""Input text on the target device. Text input widget must be active first.:param text: text to input, unicode is supported:param enter: input `Enter` keyevent after text input, default is True:return: None:platforms: Android, Windows, iOS:Example:>>> text("test")>>> text("test", enter=False)On Android, sometimes you need to click the search button after typing::>>> text("test", search=True).. seealso::Module :py:mod:`airtest.core.android.ime.YosemiteIme.code`If you want to enter other keys on the keyboard, you can use the interface::>>> text("test")>>> device().yosemite_ime.code("3")  # 3 = IME_ACTION_SEARCHRef: `Editor Action Code `_"""G.DEVICE.text(text, enter=enter, **kwargs)delay_after_operation()@logwrap
def sleep(secs=1.0):"""Set the sleep interval. It will be recorded in the report:param secs: seconds to sleep:return: None:platforms: Android, Windows, iOS:Example:>>> sleep(1)"""time.sleep(secs)@logwrap
def wait(v, timeout=None, interval=0.5, intervalfunc=None):"""Wait to match the Template on the device screen:param v: target object to wait for, Template instance:param timeout: time interval to wait for the match, default is None which is ``ST.FIND_TIMEOUT``:param interval: time interval in seconds to attempt to find a match:param intervalfunc: called after each unsuccessful attempt to find the corresponding match:raise TargetNotFoundError: raised if target is not found after the time limit expired:return: coordinates of the matched target:platforms: Android, Windows, iOS:Example:>>> wait(Template(r"tpl1606821804906.png"))  # timeout after ST.FIND_TIMEOUT>>> # find Template every 3 seconds, timeout after 120 seconds>>> wait(Template(r"tpl1606821804906.png"), timeout=120, interval=3)You can specify a callback function every time the search target fails::>>> def notfound():>>>     print("No target found")>>> wait(Template(r"tpl1607510661400.png"), intervalfunc=notfound)"""timeout = timeout or ST.FIND_TIMEOUTpos = loop_find(v, timeout=timeout, interval=interval, intervalfunc=intervalfunc)return pos@logwrap
def exists(v):"""Check whether given target exists on device screen:param v: target to be checked:return: False if target is not found, otherwise returns the coordinates of the target:platforms: Android, Windows, iOS:Example:>>> if exists(Template(r"tpl1606822430589.png")):>>>     touch(Template(r"tpl1606822430589.png"))Since ``exists()`` will return the coordinates, we can directly click on this return value to reduce one image search::>>> pos = exists(Template(r"tpl1606822430589.png"))>>> if pos:>>>     touch(pos)"""try:pos = loop_find(v, timeout=ST.FIND_TIMEOUT_TMP)except TargetNotFoundError:return Falseelse:return pos@logwrap
def find_all(v):"""Find all occurrences of the target on the device screen and return their coordinates:param v: target to find:return: list of results, [{'result': (x, y),'rectangle': ( (left_top, left_bottom, right_bottom, right_top) ),'confidence': 0.9},...]:platforms: Android, Windows, iOS:Example:>>> find_all(Template(r"tpl1607511235111.png"))[{'result': (218, 468), 'rectangle': ((149, 440), (149, 496), (288, 496), (288, 440)),

# transform.py# _*_ coding:UTF-8 _*_class TargetPos(object):"""点击目标图片的不同位置，默认为中心点01 2 34 0 67 8 9"""LEFTUP, UP, RIGHTUP = 1, 2, 3LEFT, MID, RIGHT = 4, 5, 6LEFTDOWN, DOWN, RIGHTDOWN = 7, 8, 9def getXY(self, cvret, pos):if pos == 0 or pos == self.MID:return cvret["result"]rect = cvret.get("rectangle")if not rect:print("could not get rectangle, use mid point instead") #无法获取矩形，请改用中点return cvret["result"]w = rect[2][0] - rect[0][0]h = rect[2][1] - rect[0][1]if pos == self.LEFTUP:return rect[0]elif pos == self.LEFTDOWN:return rect[1]elif pos == self.RIGHTDOWN:return rect[2]elif pos == self.RIGHTUP:return rect[3]elif pos == self.LEFT:return rect[0][0], rect[0][1] + h / 2elif pos == self.UP:return rect[0][0] + w / 2, rect[0][1]elif pos == self.RIGHT:return rect[2][0], rect[2][1] - h / 2elif pos == self.DOWN:return rect[2][0] - w / 2, rect[2][1]else:print("invalid target_pos:%s, use mid point instead" % pos)return cvret["result"]

# uiautomator.py# coding=utf-8
__author__ = 'lxn3032'import os
import requests
import time
import warnings
import threading
import atexitfrom airtest.core.api import connect_device, device as current_device
from airtest.core.android.ime import YosemiteIme
from airtest.core.error import AdbShellError, AirtestErrorfrom hrpc.client import RpcClient
from hrpc.transport.http import HttpTransport
from poco.pocofw import Poco
from poco.agent import PocoAgent
from poco.sdk.Attributor import Attributor
from poco.sdk.interfaces.screen import ScreenInterface
from poco.utils.hrpc.hierarchy import RemotePocoHierarchy
from poco.utils.airtest.input import AirtestInput
from poco.utils import six
from poco.drivers.android.utils.installation import install, uninstall__all__ = ['AndroidUiautomationPoco', 'AndroidUiautomationHelper']
this_dir = os.path.dirname(os.path.realpath(__file__))
PocoServicePackage = 'com.netease.open.pocoservice'
PocoServicePackageTest = 'com.netease.open.pocoservice.test'
UiAutomatorPackage = 'com.github.uiautomator'class AndroidRpcClient(RpcClient):def __init__(self, endpoint):self.endpoint = endpointsuper(AndroidRpcClient, self).__init__(HttpTransport)def initialize_transport(self):return HttpTransport(self.endpoint, self)# deprecated
class AttributorWrapper(Attributor):"""部分手机上仍不支持Accessibility.ACTION_SET_TEXT，使用YosemiteIme还是兼容性最好的方案这个class会hook住set_text，然后改用ime的text方法"""def __init__(self, remote, ime):self.remote = remoteself.ime = imedef getAttr(self, node, attrName):return self.remote.getAttr(node, attrName)def setAttr(self, node, attrName, attrVal):if attrName == 'text' and attrVal != '':# 先清除了再设置，虽然这样不如直接用ime的方法好，但是也能凑合用着current_val = self.remote.getAttr(node, 'text')if current_val:self.remote.setAttr(node, 'text', '')self.ime.text(attrVal)else:self.remote.setAttr(node, attrName, attrVal)class ScreenWrapper(ScreenInterface):def __init__(self, screen):super(ScreenWrapper, self).__init__()self.screen = screendef getScreen(self, width):# Android上PocoService的实现为仅返回b64编码的图像，格式固定位jpgb64img = self.screen.getScreen(width)return b64img, 'jpg'def getPortSize(self):return self.screen.getPortSize()class AndroidPocoAgent(PocoAgent):def __init__(self, endpoint, ime, use_airtest_input=False):self.client = AndroidRpcClient(endpoint)remote_poco = self.client.remote('poco-uiautomation-framework')dumper = remote_poco.dumperselector = remote_poco.selectorattributor = remote_poco.attributorhierarchy = RemotePocoHierarchy(dumper, selector, attributor)if use_airtest_input:inputer = AirtestInput()else:inputer = remote_poco.inputersuper(AndroidPocoAgent, self).__init__(hierarchy, inputer, ScreenWrapper(remote_poco.screen), None)class KeepRunningInstrumentationThread(threading.Thread):"""Keep pocoservice running"""def __init__(self, poco, port_to_ping):super(KeepRunningInstrumentationThread, self).__init__()self._stop_event = threading.Event()self.poco = pocoself.port_to_ping = port_to_pingself.daemon = Truedef stop(self):self._stop_event.set()def stopped(self):return self._stop_event.is_set()def run(self):while not self.stopped():if getattr(self.poco, "_instrument_proc", None) is not None:stdout, stderr = self.poco._instrument_proc.communicate()print('[pocoservice.apk] stdout: {}'.format(stdout))print('[pocoservice.apk] stderr: {}'.format(stderr))if not self.stopped():self.poco._start_instrument(self.port_to_ping)  # 尝试重启time.sleep(1)class AndroidUiautomationPoco(Poco):"""Poco Android implementation for testing **Android native apps**.Args:device (:py:obj:`Device`): :py:obj:`airtest.core.device.Device` instance provided by ``airtest``. leave the parameter default and the default device will be chosen. more details refer to ``airtest doc``using_proxy (:py:obj:`bool`): whether use adb forward to connect the Android device or notforce_restart (:py:obj:`bool`): whether always restart the poco-service-demo running on Android device or notoptions: see :py:class:`poco.pocofw.Poco`Examples:The simplest way to initialize AndroidUiautomationPoco instance and no matter your device network status::from poco.drivers.android.uiautomation import AndroidUiautomationPocopoco = AndroidUiautomationPoco()poco('android:id/title').click()..."""def __init__(self, device=None, using_proxy=True, force_restart=False, use_airtest_input=False, **options):# 加这个参数为了不在最新的pocounit方案中每步都截图self.screenshot_each_action = Trueif options.get('screenshot_each_action') is False:self.screenshot_each_action = Falseself.device = device or current_device()if not self.device:self.device = connect_device("Android:///")self.adb_client = self.device.adbif using_proxy:self.device_ip = self.adb_client.host or "127.0.0.1"else:self.device_ip = self.device.get_ip_address()# save current top activity (@nullable)try:current_top_activity_package = self.device.get_top_activity_name()except AirtestError as e:# 在一些极端情况下，可能获取不到top activity的信息print(e)current_top_activity_package = Noneif current_top_activity_package is not None:current_top_activity_package = current_top_activity_package.split('/')[0]# install imeself.ime = YosemiteIme(self.adb_client)# installself._instrument_proc = Noneself._install_service()# forwardif using_proxy:p0, _ = self.adb_client.setup_forward("tcp:10080")p1, _ = self.adb_client.setup_forward("tcp:10081")else:p0 = 10080p1 = 10081# startready = self._start_instrument(p0, force_restart=force_restart)if not ready:# 之前启动失败就卸载重装，现在改为尝试kill进程或卸载uiautomatorself._kill_uiautomator()ready = self._start_instrument(p0)if current_top_activity_package is not None:current_top_activity2 = self.device.get_top_activity_name()if current_top_activity2 is None or current_top_activity_package not in current_top_activity2:self.device.start_app(current_top_activity_package, activity=True)if not ready:raise RuntimeError("unable to launch AndroidUiautomationPoco")if ready:# 首次启动成功后，在后台线程里监控这个进程的状态，保持让它不退出self._keep_running_thread = KeepRunningInstrumentationThread(self, p0)self._keep_running_thread.start()endpoint = "http://{}:{}".format(self.device_ip, p1)agent = AndroidPocoAgent(endpoint, self.ime, use_airtest_input)super(AndroidUiautomationPoco, self).__init__(agent, **options)def _install_service(self):updated = install(self.adb_client, os.path.join(this_dir, 'lib', 'pocoservice-debug.apk'))return updateddef _is_running(self, package_name):"""use ps |grep to check whether the process exists:param package_name: package name(e.g., com.github.uiautomator)or regular expression(e.g., poco\|airtest\|uiautomator\|airbase):return: pid or None"""cmd = r' |echo $(grep -E {package_name})'.format(package_name=package_name)if self.device.sdk_version > 25:cmd = r'ps -A' + cmdelse:cmd = r'ps' + cmdprocesses = self.adb_client.shell(cmd).splitlines()for ps in processes:if ps:ps = ps.split()return ps[1]return Nonedef _start_instrument(self, port_to_ping, force_restart=False):if not force_restart:try:state = requests.get('http://{}:{}/uiautomation/connectionState'.format(self.device_ip, port_to_ping),timeout=10)state = state.json()if state.get('connected'):# skip starting instrumentation if UiAutomation Service already connected.return Trueexcept:passif self._instrument_proc is not None:if self._instrument_proc.poll() is None:self._instrument_proc.kill()self._instrument_proc = Noneready = False# self.adb_client.shell(['am', 'force-stop', PocoServicePackage])# 启动instrument之前，先把主类activity启动起来，不然instrumentation可能失败self.adb_client.shell('am start -n {}/.TestActivity'.format(PocoServicePackage))instrumentation_cmd = ['am', 'instrument', '-w', '-e', 'debug', 'false', '-e', 'class','{}.InstrumentedTestAsLauncher'.format(PocoServicePackage),'{}/androidx.test.runner.AndroidJUnitRunner'.format(PocoServicePackage)]self._instrument_proc = self.adb_client.start_shell(instrumentation_cmd)def cleanup_proc(proc):def wrapped():try:proc.kill()except:passreturn wrappedatexit.register(cleanup_proc(self._instrument_proc))time.sleep(2)for i in range(10):try:requests.get('http://{}:{}'.format(self.device_ip, port_to_ping), timeout=10)ready = Truebreakexcept requests.exceptions.Timeout:breakexcept requests.exceptions.ConnectionError:if self._instrument_proc.poll() is not None:warnings.warn("[pocoservice.apk] instrumentation test server process is no longer alive")stdout = self._instrument_proc.stdout.read()stderr = self._instrument_proc.stderr.read()print('[pocoservice.apk] stdout: {}'.format(stdout))print('[pocoservice.apk] stderr: {}'.format(stderr))time.sleep(1)print("still waiting for uiautomation ready.")self.adb_client.shell('am start -n {}/.TestActivity'.format(PocoServicePackage))continuereturn readydef _kill_uiautomator(self):"""poco-service无法与其他instrument启动的apk同时存在，因此在启动前，需要杀掉一些可能的进程：比如 io.appium.uiautomator2.server, com.github.uiautomator, com.netease.open.pocoservice等:return:"""pid = self._is_running("uiautomator")if pid:warnings.warn('{} should not run together with "uiautomator". "uiautomator" will be killed.'.format(self.__class__.__name__))self.adb_client.shell(['am', 'force-stop', PocoServicePackage])try:self.adb_client.shell(['kill', pid])except AdbShellError:# 没有root权限uninstall(self.adb_client, UiAutomatorPackage)def on_pre_action(self, action, ui, args):if self.screenshot_each_action:# airteset log用from airtest.core.api import snapshotmsg = repr(ui)if not isinstance(msg, six.text_type):msg = msg.decode('utf-8')snapshot(msg=msg)def stop_running(self):print('[pocoservice.apk] stopping PocoService')self._keep_running_thread.stop()self._keep_running_thread.join(3)self.adb_client.shell(['am', 'force-stop', PocoServicePackage])class AndroidUiautomationHelper(object):_nuis = {}@classmethoddef get_instance(cls, device):"""This is only a slot to store and get already initialized poco instance rather than initializing again. You cansimply pass the ``current device instance`` provided by ``airtest`` to get the AndroidUiautomationPoco instance.If no such AndroidUiautomationPoco instance, a new instance will be created and stored. Args:device (:py:obj:`airtest.core.device.Device`): more details refer to ``airtest doc``Returns:poco instance"""if cls._nuis.get(device) is None:cls._nuis[device] = AndroidUiautomationPoco(device)return cls._nuis[device]

从touch图片来说，通过loop_find去循环找图，找到图片的话会返回坐标，后面的代码会去点击这个坐标，超时未找到报错。就完成了touch操作。

loop_find的识别阈值threshold范围是[0.0, 1.0]，默认0.7，就是相似度，当识别可信度=>0.7时就认为是匹配的。当识别出一张图像后，Airtest将会默认去点击图像的正中心位置，有时我们希望它识别出图片后点击其他位置，可以通过修改target _pos属性来实现。target_pos取值范围是1~9，[1,9]，且必须为整数，默认值是5(图像正中心)。

loop_find识别并返回坐标数据：
1、首先会获取手机屏幕截图；
2、然后对比脚本传入图片获取匹配上的位置；
3、一直重复上面两步骤直到匹配上或者超时。

根据已知模式到另一幅图中寻找相应模式的处理方法就叫做模板匹配。 模板就是一幅已知的小图像。模板匹配就是在一幅大图像中搜寻目标。已知该图中有要找的目标，通过一联的算法可以在图中找到目标，确定其坐标位置。

匹配的是调用Template对象的match_in，传入截图，返回坐标点。
match_in方法调用_cv_match-->_try_match
_cv_match方法是读取图像进行模板匹配，循环各种算法来去匹配图片，匹配到之后就会直接返回结果。（传入的图像需要进行缩放变化，提高匹配成功率。）
CVSTRATEGY用于设置Airtest的图像识别算法，默认情况下CVSTRATEGY = ["surf', "tp" "brisk”]，每次查找图片的时候，airtest就会按照这个设置好的算法顺序去执行，直到找出一个符合设定阈值的识别结果，或者是一直按照这个算法顺序循环查找，直到超时。

find_best_result只筛选出最优区域：
1、输入校验图像，2、计算模板匹配的结果矩阵res 3、依次获取匹配结果，4、求取可信度:如果有颜色校验,对目标区域进行BGR三通道校验 5、求取识别位置: 目标中心 + 目标区域:根据左上角点和宽高求出目标区域

cv2.matchTemplate是opencv的方法，它的返回值是个矩阵，相当于用小图在大图上滑动，从左上角开始，每次移动一个像素，然后计算一个匹配结果，最终形成结果矩阵。灰度识别: cv2.matchTemplate( )只能处理灰度图片参数。匹配方法:TM_CCOEFF_NORMED计算归一化相关系数，计算出来的值越接近1越相关。

match_all_in这个方法匹配的是多个，调用_find_all_template-->TemplateMatching.py
调用的都是find_all_results方法，进行算法匹配，返回结果
find_all_results返回所有大于指定置信度的结果，多了一个：屏蔽已经取出的最优结果,进入下轮循环继续寻找。

TargetPos 图像匹配（大图中找小图）

TM_CCOEFF_NORMED： 该方法名为归一化系数匹配法 这种方法将系数匹配方法进行归一化，使得输入结果缩放到了1到-1之间，当模板与滑动窗口完全匹配时计算数值为1，当两者完全不匹配时计算结果为-1。由于matchTemplate()函数的输出结果是存有相关性系数的矩阵，因此需要通过minMaxLoc()函数去寻找输入矩阵中的最大值或者最小值，进而确定模板匹配的结果。通过寻找输出矩阵的最大值或者最小值得到的只是一个像素点，需要以该像素点为矩形区域的左上角，绘制与模板图像同尺寸的矩形框，标记出最终匹配的结果。

总结：程序中采用TM_CCOEFF_NORMED方法计算相关性系数，通过minMaxLoc()函数寻找相关性系数中的最大值，确定最佳匹配值的像素点坐标，之后在原图中绘制出与模板最佳匹配区域的范围。

find_template函数是返回第一个最匹配的结果（位置未必在最上面），而find_all_template是返回所有大于指定置信度的结果。

cv2.matchTemplate是opencv的方法，它的返回值是个矩阵，相当于用小图在大图上滑动，从左上角开始，每次移动一个像素，然后计算一个匹配结果，最终形成结果矩阵。 结果矩阵大小应该是: (W - w + 1) x (H - h + 1)，其中W,H是大图的宽高, w和h是小图的宽高。 这个矩阵中最大值的那个点，就表示小图的左上角对在这个位置时，匹配度最高，由此得到第一个匹配结果。

opencv的matchTemplate本来只支持灰度图，把bgr三个彩色通道做了分离，分别调用matchTemplate，然后再合并结果。

原理

selenium底层原理

1.selenium client(python等语言编写的自动化测试脚本)初始化一个service服务，通过Webdriver启动浏览器驱动程序chromedriver.exe

2.通过RemoteWebDriver向浏览器驱动程序发送HTTP请求，浏览器驱动程序解析请求，打开浏览器，并获得sessionid，如果再次对浏览器操作需携带此id

3.打开浏览器，绑定特定的端口，把启动后的浏览器作为webdriver的remote server

3.打开浏览器后，所有的selenium的操作(访问地址，查找元素等)均通过RemoteConnection链接到remote server，然后使用execute方法调用_request方法通过urlib3向remote server发送请求

4.浏览器通过请求的内容执行对应动作

5.浏览器再把执行的动作结果通过浏览器驱动程序返回给测试脚本

appnium底层原理

在Android端，appium基于WebDriver协议，利用Bootstrap.jar，最后通过调⽤用UiAutomator的命令，实现App的自动化测试。

UiAutomator测试框架是Android SDK自带的App UI自动化测试Java库。

另外由于UiAutomator对H5的支持有限，appium引入了chromedriver以及safaridriver等来实现基于H5的自动化。

appium 在android端工作流

client端也就是我们 test script是我们的webdriver测试脚本。

中间是起的Appium的服务，Appium在服务端起了一个Server（4723端口），跟selenium Webdriver测试框架类似， Appium⽀持标准的WebDriver JSONWireProtocol。在这里提供它提供了一套REST的接口,Appium Server接收web driver client标准rest请求，解析请求内容，调⽤用对应的框架响应操作。

appium server会把请求转发给中间件Bootstrap.jar ，它是用java写的，安装在手机上.Bootstrap监听4724端口并接收appium 的命令，最终通过调⽤用UiAutomator的命令来实现。

最后Bootstrap将执行的结果返回给appium server。

appium server再将结果返回给 appium client。

Airtest

底层使用opencv来做图像算法

逻辑和appium基本一致

参考于：

【airtest】airtest学习汇总--入门篇 - 走看看

python - 图像匹配（大图中找小图）之find_template源码解析_个人文章 - SegmentFault 思否

OpenCV中的图像基本操作--B站视频教程笔记(七)_走丢的大白的博客-CSDN博客

【从零学习OpenCV 4】图像模板匹配 - 知乎

 Airtest图像识别 - 腾讯云开发者社区-腾讯云

airtest源码分析--图像识别整体流程 - 简书

Airtest图像识别原理 - 简书

Airtest基础使用原理和airtest中touch定位说明 - 百度文库

Airtest命令行运行airtest run详解

 Airtest的多机协作、交互及设备切换set_current() - ☆星空物语☆ - 博客园

Selenium和Appnium、Airtest的底层原理-阿里云开发者社区