在Appium操作时弹出普通窗口以及输入密码时的随机软键盘,由于焦点没有切换过来,或者整个软键盘是一个整体,导致无法直接通过id,xpath定位,这个时候只能利用图片识别来解决问题.
思路:
1. 保存要识别的图片A
2. App运行时,到了目标页面,截屏 self.driver.get_screenshot_as_file('/Users/xxx/test/jianpan.jpg')
3. 对比,寻找图片A的坐标
4. 点击
操作:
- 安装opencv库
pip3 install opencv-python - 方法代码如下:
import cv2
import numpy as np
import time
def get_pay_keyboard_number_location(self,impath, target,fit_num): #fit_num是匹配度,如 0.95,0.85
print("start find pic")
positions = {}
start = time.time()
img_rgb = cv2.imread(impath)
teNum = "done"
template = cv2.imread(target)
h, w = template.shape[:-1]
res = cv2.matchTemplate(img_rgb, template, cv2.TM_CCOEFF_NORMED)
threshold = fit_num # 匹配度参数，1为完全匹配
loc = np.where(res >= threshold)
if len(loc) > 0:
# positions[teNum] = zip(*loc[::-1])[0] # python2的写法
print("找到了匹配图")
zipped = zip(*loc[::-1]) #list[::-1] 相当于起点为最后的一个,终点为第一个,然后一次减少一个
positions[teNum] = (list(zipped))[0]
else:
print("Can not found pic: ")
end = time.time()
print(end - start)
return positions[teNum]
- 调用代码:
impath1 = "/Users/xxx/2019work/eebank/pic/jianpan1.png"
targetPath_a = "/Users/xxx/2019work/eebank/pic/a.png"
ls = self.get_pay_keyboard_number_location(impath1, targetPath_a,0.85) #在键盘图片找到字母a
# ls = (62,1046) 这样的数组
dd = ls[0]
kk = ls[1]
m = (dd/2, kk / 2)
#点击
self.driver.tap([m],20)
