[Fuego] [PATCH 09/14] Add mask-img-path argument to CheckScreenshot for ignored areas
Tim.Bird at sony.com
Tim.Bird at sony.com
Thu Apr 26 18:07:15 UTC 2018
> -----Original Message-----
> From: Guilherme Camargo
>
> On Wed, Apr 25, 2018 at 8:28 PM, <Tim.Bird at sony.com
> <mailto:Tim.Bird at sony.com> > wrote:
>
>
> > -----Original Message-----
> > From: Guilherme Campos Camargo
> >
> > The new mask-img-path argument that has been added to
> CheckScreenshot
> > can be used to provide a mask image that will determine which
> parts of
> > the screenshots will be compared.
> >
> > The mask needs to be a Black and White .png image in which the
> Black
> > areas will be ignored on the comparison, while the White areas will
> be
> > compared normally.
>
> How do you generate these?
>
>
>
> Actually, it's pretty straightforward with Gimp (or any other image editor with
> layers).
>
> I just add a new transparent layer above the original image, paint black the
> areas
> that I want to remove (in the new layer) - right on top of the original image -
> then
> add a new white layer in between the two and export.
That seems pretty easy.
>
>
> It seems like it would be nice to be able to use a chroma-key color
> (like magenta),
> for the to-ignore regions, and have all other colors be interpreted as
> white,
> for the "to-compare" regions. This would allow creation of the masks
> by
> just drawing a rectangle of magenta on the reference image.
> That is, you could pull in the mask image in the above-described
> format, and
> convert all colors but magenta to white, and magenta to black, and
> then
> use the mask image like you are doing now.
>
>
> That's an option, but the problem with that is that magenta would need to
> be a prohibited color on the original image.
Yeah, but magenta is a fairly seldom-used color, and even if some other screen
pixels ended up not being compared, I don't think it would be a problem.
Most images are not going to have contiguous blocks of magenta in them.
>
>
>
>
> But maybe you have a workflow where it's already easy to create the
> masks.
>
> One good thing about this is you could look at the mask and more
> easily tell
> what part of the image you intend to ignore.
Having the rectangle mask in the context of the original image, viewable without
a special tool, seems like it would be nice.
However, this was just brainstorming on my part. If I find myself making lots
of masks and I think the workflow is cumbersome, I'll just try some things myself, to see
if it has the advantages I'm postulating.
> -- Tim
>
>
> >
> > A new CheckScreenshot has been added to check the
> 'page_generated'
> > element of the Jenkins web-interface. That element is positioned
> in the
> > lower right corner of the page and contains a DateTime field that
> > changes continually.
> >
> > The assets used as reference and mask have also been included in
> this
> > commit.
> >
> > Signed-off-by: Guilherme Campos Camargo
> <guicc at profusion.mobi <mailto:guicc at profusion.mobi> >
> > ---
> > .../screenshots/page-generated-mask.png | Bin 0 -> 183 bytes
> > .../screenshots/page-generated.png | Bin 0 -> 4583 bytes
> > .../Functional.fuego_release_test/test_run.py | 25
> +++++++++++++-----
> > 3 files changed, 19 insertions(+), 6 deletions(-)
> > create mode 100644
> > engine/tests/Functional.fuego_release_test/screenshots/page-
> generated-
> > mask.png
> > create mode 100644
> > engine/tests/Functional.fuego_release_test/screenshots/page-
> > generated.png
> >
> > diff --git
> a/engine/tests/Functional.fuego_release_test/screenshots/page-
> > generated-mask.png
> > b/engine/tests/Functional.fuego_release_test/screenshots/page-
> > generated-mask.png
> > new file mode 100644
> > index
> >
> 0000000000000000000000000000000000000000..a565ba4dd4b8cbd5ab4be9c70
> > 50d8046f63e238a
> > GIT binary patch
> > literal 183
> > zcmeAS at N?(olHy`uVBq!ia0y~yU{nCI`8e2sWPi}XA|S<<<n8Xl at E-
> > &h>|H(?D8gCb
> > z5n0T at z%2~Ij105pNB{-
> > dOFVsD*&ne8i79f=*0Tx$3b}Z?IEG~0dwXRgFM|Qk;SIa~
> > z|9=xV!PeoVu!%-)b%bgsN6kYo1_^-
> > #2L?7434sPiMmClN2OtF(I`D*X)n68A7Wv8g
> > QK=T+pUHx3vIVCg!0HVY$sQ>@~
> >
> > literal 0
> > HcmV?d00001
> >
> > diff --git
> a/engine/tests/Functional.fuego_release_test/screenshots/page-
> > generated.png
> > b/engine/tests/Functional.fuego_release_test/screenshots/page-
> > generated.png
> > new file mode 100644
> > index
>
> >
> 0000000000000000000000000000000000000000..9b16d92e5cc3ec6c9f57723115
>
> > e21f8f23f5eab9
> > GIT binary patch
> > literal 4583
> >
> zcmV<D5g6`?P)<h;3K|Lk000e1NJLTq00AHX000gM0ssI205Dc1000rLNkl<Zc%1E
> > 8
> >
> zXF!ulx1Kizl2Aek5Q>6yWEH^%Hae&v0wOEc#j at BC*Ig?jNE5pvh=LRmP?|~+0
> > RfS2
> > z`>d-
> VDxDBI2}vLcy!Xer*+^6O?sxCE_xk+doik^iIp<6}GXY_98Q?p<<2(L6zyJWy
> >
> z&+)%)(%+A7^^47BFPdjDz!~`4nI|9_&GSuzGM at 67^1s~U1hrq;a6C&U_VJ7T
> > ezDUD
> >
> z^e12*Qw<#p<GG0g0EC_kPfJM$0FX!|Ep5%6UfXqav=IcEz~O%ua_;1o6qkgY4
> > F21l
> > z+T*pi`fbha>@-
> > nP(Z9yf(cW?D#F at 8ms(X5R(=$@!<>jE9#bTW~dG2A=qoJYU8Ab+s
> >
> ze%mc4M;VXg at uMf#t|q>#sm0 at Q#wK%jdTl3@$*?Lm`s&TJTYUbIm5t at D-
> > 8)1?L;wIG
> > zK__0kc-
> >
> 7L};^XISZDT12OMF~H#_cR7gQ>1D#nWrM{tx;iB6i}~sprpMwludKIqGd?
> >
> zZ7GnKm6V5vUFvA>Fqkp at p!Yt?C{xo at Z{Eqe+uYnjmLe~6uv at ob4F-
> > c at v)SGUeBRf7
> >
> z;BW?1RaG~8xEq_y9X&>6CFK{xFVoxTGYkz59NH`QM~DCbv{u@>)$Rr}rk^@{
> > d_=?h
> >
> zg~h%;{wRv#a5!aUrJpvfx3aeE?d@||>ZGWocs=DB27`fdcC>fF;cz+mnIjl%ES8=
> > +
> >
> zd)#2gbXZKekycgt at JF|m`+hw*dO at b&PR5IfL`TJzmsK$O7*s_CODl`jYgb`lwV
> > nNv
> > z`=$4i)032yl>+>NhKGm88_WM#peQ;tH1t2&BdwM8;`vJiLCQ-
> > jCf<OB!RPaZiwFi|
> > zGSAq@&wIq^UX8t0U*8xTe<e9R5k-
> > e1!Y&UFjXNmq?H%?GORpwGU$}U_si`?UGy;xb
> > ze%`&z+gazr&Rk81e*N-
> > w;<f8guC1fB at 4&B<RVNAeSyobhJux{b#Q#oShOx<9AMc~1
> > z<Ef>s`P;r<RMjR63tL-
> > S1C9o7+Wgba%oH+NGSL6n=xB6xb#L40aU(79(Bb{*sW<QD
> >
> z<P8rG<8ZjO>(|6yi%z?p?6%g`_ppC&Z|{hPtu3tq{=pkJZMdDCMwXHc3J4LNo
> > M+F3
> > z>gZ~Z7Zzm|<-
> > !MfIr*6`uFfGrC+Tf;D92(kFJHWZMm{Jmm6IQL=NgSUb7$YXf487G
> > z_e$Jl3<i^vpIK0xTTq-UCNAc=-
> > HX;r3qBr at nR7EZB#^;i*1mg>gUt{G!QpUHQsm`M
> > z4!ODc91bTnIX&l2E~Ag3tgN(S*EXY>h6sWT4-
> > ZF2#b#w>ixLQH*RMW(;%sg~Hi<;)
> >
> z>gqm!=EC!5&qWD at 6)sM$t6bprAOM5GI1zIC+0*B8auhQQ)5?mf%U3P}0IJ_
> > shlNJe
> > zeXN(0qioyZG1p`c0Km^DKuVh2P~X_q*(D(<;d}J3yn_71%Jp>(tu$I-
> @X^u-
> > Wx03r
> >
> zh0ShmYKgjZrN5u$x?(k1QZh0og3sq)jlFg+zhHnfuwdbQ&)qwSL?Vi!F<0WUGP
> > 3a^
> >
> zBI`D+5loSH?`kW{tKg11BG&nN1$t9;b at g=ea`VkC&7jcVH;^PI*4)(0WH8B6<oy
> > SK
> >
> zQ=lp|HZ*x|_i|d{SX}f#OI!1h&jG=5qEKWVotO9a_KhHKYojk%IFBMnK at h~k(
> > mXjS
> >
> z6$Rt2VLSUJf^QdB=lHk;Xn$@_{&FV=4Gnb!K~}GGi;Ibej;vVe3;@7kal#}tTAPlp
> >
> zwyv%Yf*|(GmR^m$#$+<3rKN>+oL!s%0C*f;SXfwaUvH|exuqEvi}l#HWu>zly
> > ^T&)
> >
> zqzZ at 5W7`&}EGsLs$aG=#+nU8oY(zvvX3jE#rm9R*;c~g19bMAW(kvD$E+&5U
> > S~qEF
> >
> zsltN$dV0EM7N$5H&SUGAmCkPM^meKu6#!6HQjWo3Oy(Is``7dFB7?)>L_|a!
> > otH<1
> >
> zN7mKV>+0zO0IY4S^79I&8%#qG<X&EZ?P8l7Nh#woi~t6Mf!PWUhXV+}<MF
> > VVnw-w%
> >
> z@`6JA2?PQFKuk<*_a09aML(M~!^6W5ic93=<cLJ#WYtNRqQZ0XGaXkf_xBCta
> > 5w;f
> > z{Jeq(#U)WOm*Nv+o;-
> > dETcIfGd)QBbDxZ4m#)a^+S((|D<qt>fVd0?>0D#n6*F#PQ
> >
> z6&2ox!tU;#y?YKgIWNz+lfL`cUBQ7NU0vO<s;csl_mP9~NwHJ)_2OdUCsr;itvG
> > ae
> > ze at a@CmX at Z#Y$q2-
> > f8Rhhn>}f=%9gF0XU;NANl${iPFzg<@2}s4MT91&Co!1Jm at 9E8
> > zist7P6c*eMzZ7~kA-b%zTwr-
> > qV^d~E_LxWdz5GH8OEYU*t49x?2#f&$R8~|SIeIwy
> > zYNXLj!(%}wU|R-
> *F*GzBmk=$yx+cIf2fN3Qo^*D0F&K=Zg8OrfXMbgW`})-
> > zH8nN(
> >
> z{qbX+mX;=LsG+Ie+R{2WI5=KIGxJ5=UES5yHC!$?J1b|ZzMiC{<k$4rP~WJbsSb
> > Nl
> >
> zpem5blJ)fsD2k35Iuu3UzNwxvMOEOA*Umk5w#&D;Z#9{3tgbNy0Knl4BwtTu
> > G8ki%
> >
> z*EOIh%HeP#FJJN8wf$=>tSGPIa=9uh%CKl|VOCjQIXF1jMr)(d+Th{wH9QmG$
> > >XOB
> > z7tJ4WQ$P^p(*nDfU%1 at P+1Apow505y_dW~;V`*g}D=UM=VjY&-
> > ixLP;P0c8Z78eye
> > zIyop%6^KOQx(#cfSsy-ptgCC-
> > y8UNSQBkTQ)p><u$%6+Y7TEow2OHei5s5^qBGqB}
> >
> zGAJy(S2R^$&)m`shr`V<G%zqUcv$raR at vBE$;e0p0Orh{T~kv#v9h&|B~1De1h
> > KZU
> >
> zf at UvsuoEQ^nwwijrz=?*cQ*g*E+<DJ5{d4cHWc44K~c1*pvck5K~YJOL?XFwT
> > rY^Y
> > zrIkhc?PMGdH-hBV%h$A4nyHzouC9)<ic&#-
> > p`as6D|2YPi>veVXD at nsdQlVw2=b$w
> > z3pDa;224X!gG3 at ZJFK*`UEb5vw`QH&=p%GADl4iA?-
> > g$T*&Qkma0W<XBv>UbCI$fD
> > zjQ?^YEhB9*-
> > `Hc*c1u&+jN4hi{Oa{JJpusO95#tW5){S6#n~)27$<a5m!mNl%o4lB
> > z0&@Q#zo?jqZ96<>%`$?)p-
> > ^N?DvBnnP8K3Z{v>C!0RZuF31;S|%73hxF+ppkIW2Qp
> > zV!bTLKX`}dR+ULAP%cT9)SsqbRq-
> > ${_nx_h=~ucrF&KRe3T2E`^ON(7Oc!qV+$ttU
> >
> zA`*#El6yBlJ at r<1cQ*!u>Fnt2>+J&obar+LSmfp9;Rl^Y=MHi=tlbEUJRV;|Q(d?Z
> >
> zCX+cZFrc6y55p(8tkdcA*RS5JU*iso9L~Vhsd}G6ASnTr2?WBx0B2%l85wE8tj)Q
> > T
> > zdn-
> NTPqTYQre6L1{Y)k^C?Euj#R33O6y**M0suNYx~NnI$fKyF^u>VY-
> > OZml%TP*+
> > zjH0Nel||m&eAiVju(=?uA`l2-
> > Vq%@0odi*Wgt+)OJ%k(&J}Dw9l6Chs9*>K>6n*%h
> > z at 9A?Vpv~jqUr(Qf1f2*9379l_(x>PV32?3h0Qy-
> > h005Cl7_a$_q?FQ<viQVUWfi5O
> > z!s5Mq_D5fhl$DkJqNyYzNze-bfYskGSk)tta4o5<wEWDuQ-
> > Wy*0FaU*OG%MeuU)la
> >
> z?M8JCbwi^WBgk1S7WgAQhU9f0>q|?^E=7h5Ki`esx#Sh(!y?WT2%@8pVq075
> > oSfXo
> > zhDQHDzkz|VHys=f$LI3}%zQq7^gb&9S&H1z(J=-
> > I{39wVLZSQ_%BIHV at X&~JVW%}T
> > z)KL^&wQ?==ghHVRSm^XmS4KGnISEPe_{7-pb^-
> > |r2_li$)<&0*kU$Wmz5Nq|yuAF(
> > zSw?>TJ{T<gwE#@8xlf=hh-
> > `CH%Y}2{VVBNBv)x?R!F!8v=*7jwB_t&L1N}6$G at u+t
> > zP)?rG-
> >
> O~fVyM^*SFxKF4IGp=M5BPk(i_?z?f^fM!CX at N@U9FCeHUNM|YlCXNy}kYY
> >
> z{j##MtbW$FCLwio^}BZOl#(I?0300cH*NTt$KzqKSU3=)*`7apar8*Qk)wyFP5(h
> > q
> > ztD&J at Tl;?g!g&aSe0cvsNm*%vMgH-
> > _$N3B9sj5vz5X8pT>f(in_q88nWMsZ*x!M%f
> > zCy$>F4-
> >
> aE77<wDMua7Zhikfh2;YafI)ST?xbD^gwa(~V=VFeULX|y)O&*f at URG&Wi
> > z7xbFmMrSaXYHF&F9zO2s=w7>O1A-uIHiyUK{^+tMIW2L-
> > 48&rw6pHMa)6#g}+%u=o
> > zQ7Ce%YLjcK-;L=Pm8#I%N;8^i2<>TYX;q*qj2DDivu9 at 9&iZ-
> > lW>HZB3>dU|>?OhG
> >
> zuth{gV9r`mR!M86LAj~<qKw;F?eumIhm#PW1SvE%H53)8mo7%K+3caAp}LRt
> > Z+?F>
> >
> zA~7>HTNodgz~OM{ZS<_nY$&v_GJo;>MPXqPkH_P3xi4S56uk3{#@8sb*ep?k
> > C^WmW
> >
> zyo%OJgH}jMOSRGHg6BJXhozS;Mz+z~005mGoe!%Xp(tu*VRrXUF1%`{rKAh&
> > udRI_
> >
> zAD1xVB2ivefgnh1eAJb=$f(%MR}*8V8%!(6`;;XX7ZumkyyNkBS0ZC)7#av4uw!
> > Cy
> >
> zaFEC40s#0tp5Tt8tEZEDHxC~ESs8aGPo9Lw;{gDtPMi(!3mOsLH^0C2KH&4)zCA
> > xo
> > z)8}%zTrL;NZ5CT+W at I<iH+FV*-AGKawy}h>jmP5+4i2Iy%I6Oa4h}-
> ^rt0felvlPi
> > zw+syp6&4ir_p>Jbky8mUW<zre)9SZ16=jty7Ara`)@Y_7e3TLn-
> > OaRHDamQajt7#Z
> >
> z$b*A}a4f3dR+ki)c6N4kc6P<Z#M4^a^!4=s04x?OBK&esPcI~Y`=+|0tdh-
> > UM at PjP
> >
> z8qJVbkhfWEeeHT248Du2^R(&HA|k`SDbvH_ at yAac^YJ}2K~i90WuAOJ<^B5)0
> > |Nt(
> >
> zA3Z55EH*V=IG(bjvjZNF^Eu>OQ&YoYu^Q?d&z}u@{p$BWUlE11b#%0?Z7est
> > dr+xV
> >
> zO)U*|^(j!ncCmGHQ}bqb4^aZaZMADjaVZXm!(cE0!AHYGBUZbv<8pbECaY}R{
> > L_f;
> >
> zckJ4B^7xsRPHu8?6jQTB at Y#n#kv)0#ctqIcbEhw0u^1g)?LEKkhQ<inBgo6Ys4O
> > sB
> >
> zK|w)FTXTxK8myT$%jlMZyraE~grr1La{R{4KPAK`{o=K^x2IQ5o?^euZm#hh8(X
> > WU
> > z#-
> <$}yQF2L=Pxip5Co?EjSY=inRiB9f%0?jEnaFvRip}pD^@y%o(<cwbrXt$B}*5d
> > zKXak(W4(^9HoVaO;l&usjL+v=n}2!(T(xp7002JBdi~;g{_KU-
> uIq+|hPAXceU5rV
> > zo{si*lX=F-=TzC5cbLq6?}JBRQCwU+Gv_7_hqGO5-
> > O}9Rx!nu?4kqwox96{~UcLqZ
> >
> z91A>tEbw^9i6E2t#!f37J32ad?cCGP>{nEz`uHAFR8kx<G4^=vg%8}p0U^NwA;(
> > Vz
> > z8=K5kR#Eo$IdC!jaz}f|3`2v%M-IYP!lAoyJ+-
> r=bKUAs6Jy>2lb~b%2nLgsc%$xP
> >
> zJr;{qo1z*J><8~%@R7rQnVqz>l(LHQ;Ufpb!!C9Bbr{T;?(K5`gTW981o&7703eY_
> >
> zcsyQKR%S$CKc7UO$>E2#w$>QcLU3F2Bag>Be9*VMt6ND)(R25XnX`>P;};qQF
> > )^{Q
> >
> zh|nuhF^Bg1^!4^BDp4)1EHt$=kZ=80DZ$xYU0ri%zfbDT>l0rLMkD!p>bsg+AHP
> > H2
> >
> z)asuCf8W5V`g$&|EB+niLqkLE>o;GCy$pXZ{*M2AeB1j8kH>ph^=No_n8{?uM
> > 8z&x
> >
> zIR7iJ7EO)KA3l6UQM9q4>DJAR1q<i>C2Rk`1B=DRC&qqXUH>4A`zkUfP!x at gj`
> > #Ns
> > z!sBtX=givTvGFStUlyz1-
> > !Hhgr<Wu~va?@eW?}l5to=ve|IWXdeaC+v{{{QoPnL8U
> > RPL==w002ovPDHLkV1i3q$F=|f
> >
> > literal 0
> > HcmV?d00001
> >
> > diff --git
> a/engine/tests/Functional.fuego_release_test/test_run.py
> > b/engine/tests/Functional.fuego_release_test/test_run.py
> > index ba840b6..5fa2ca1 100755
> > --- a/engine/tests/Functional.fuego_release_test/test_run.py
> > +++ b/engine/tests/Functional.fuego_release_test/test_run.py
> > @@ -153,7 +153,7 @@ class CheckText(SeleniumCommand):
> >
> >
> > class CheckScreenshot(SeleniumCommand):
> > - def __init__(self, locator, pattern, ref_img, diff_img=None,
> > + def __init__(self, locator, pattern, ref_img, mask_img=None,
> > diff_img=None,
> > expected_result=True, threshold=0.0,
> > rm_images_on_success=True):
> > def add_suffix(filename, suffix):
> > @@ -166,6 +166,7 @@ class
> CheckScreenshot(SeleniumCommand):
> > self.pattern = pattern
> > self.locator = locator
> > self.reference_img_path = ref_img
> > + self.mask_img_path = mask_img
> > self.expected_result = expected_result
> > self.threshold = threshold
> > self.rm_images_on_success = rm_images_on_success
> > @@ -178,8 +179,8 @@ class
> CheckScreenshot(SeleniumCommand):
> > self.test_img_path = add_suffix(ref_img, 'test')
> >
> > def compare_images(self, current_img_path,
> reference_img_path,
> > - diff_img_path, threshold):
> > - cmd = ['magick',
> > + mask_img_path, diff_img_path, threshold):
> > + cmd = ('magick',
> > 'compare',
> > '-verbose',
> > '-metric',
> > @@ -188,10 +189,14 @@ class
> CheckScreenshot(SeleniumCommand):
> > 'Red',
> > '-compose',
> > 'Src',
> > + '-read-mask' if mask_img_path else None,
> > + mask_img_path if mask_img_path else None,
> > current_img_path,
> > reference_img_path,
> > - diff_img_path
> > - ]
> > + diff_img_path,
> > + )
> > +
> > + cmd = list(filter(None, cmd))
> >
> > LOGGER.debug(' Comparing images...')
> > LOGGER.debug(' cmd: $ %s', ' '.join(cmd))
> > @@ -246,6 +251,7 @@ class
> CheckScreenshot(SeleniumCommand):
> >
> > result = self.compare_images(self.test_img_path,
> > self.reference_img_path,
> > + self.mask_img_path,
> > self.diff_img_path,
> > self.threshold)
> >
> > @@ -743,7 +749,14 @@ def main():
> > CheckScreenshot(By.ID, 'tasks',
> > rm_images_on_success=True,
> > ref_img='screenshots/side-panel-tasks.png',
> > - threshold=0.1)
> > + threshold=0.1),
> > +
> > + # Compare screenshot of an element of Jenkins UI ignoring an
> area
> > + CheckScreenshot(By.CLASS_NAME, 'page_generated',
> > + rm_images_on_success=False,
> > + mask_img='screenshots/page-generated-mask.png',
> > + ref_img='screenshots/page-generated.png',
> > + threshold=0.1),
> > ]
> >
> > if not execute_tests(args.timeout):
> > --
> > 2.17.0
> >
>
> > _______________________________________________
> > Fuego mailing list
> > Fuego at lists.linuxfoundation.org
> <mailto:Fuego at lists.linuxfoundation.org>
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
> <https://lists.linuxfoundation.org/mailman/listinfo/fuego>
>
>
More information about the Fuego
mailing list