A tool for testing adaptive layout. How to test responsive design for free. Mobile emulator from Google DevTools

One of the most noteworthy tasks that the EastBanc Technologies QA department has ever faced is the creation of an automated testing system for the website www.washingtonpost.com. This is an electronic newspaper implemented as an information and news portal.

The main reason for the need to create an automated testing system was that the application was planned to transition to a new CMS (the so-called PageBuilder), which should replace several other CMSs previously used to publish content in various sections of the site. With this kind of migration, it is very important to avoid mistakes so that the content published through the new CMS on various types of pages looks appropriate.

We are not faced with the task of checking all pages for compliance with our tests. Our task is to identify PageBuilder bugs, check the reliability of the layout of pages created by a freshly baked PageBuilder, and draw the attention of Washington Post editors to those nuances of filling a specific page with content that may lead to potential problems in displaying pages.
The creation of a testing system is under active development, but some points that we think are interesting can already be presented to the general public.

Before we do this, it is necessary to note one feature of the project: all our testing takes place “outside”. Those. We, like any other user, use the combat version of the site for testing.

Having explored the Internet, we settled on the following approaches and tools. To test the page framework, we adopted the Galen framework, which we then integrated with testNG.

Naturally, passing the Galen test for a page frame does not mean the layout is valid. In addition to the location of the blocks, you also need to check the display of various elements within the block. We decided to test the internal contents of the blocks by comparing screenshots.
Screenshot tests include various logos, buttons, some blocks with specific display - everything that Galen cannot reach and that is difficult/impossible to check with functional tests.

Azure color - tested by Galen, filled with green - screenshot tests:

Carefully! Big picture

Hidden text

Galen and Screenshot tests can successfully replace some functional tests, sometimes gaining in clarity, sometimes in speed, and sometimes in both. We select a testing method for a specific case through a collective discussion of a test case for each type of page based on the criteria of performance, ease of support, completeness of test coverage and clarity.

For example, there are 2 blocks that we initially wrote functional tests to test: Most Read and Information Block. Now we check the first one with screenshots, and the second one with a Galen test.

MostRead Block, screenshot test:


Regarding the functional test: there are significantly fewer lines of code, the completeness of test coverage has increased, and the test is updated when changes appearance this block on the page will not take much time.

Testing this block is discussed in the chapter on the screenshot method.

WaPo Information Block:


Galen easily copes with checking the correspondence of the text and links of a given block: the links themselves are specified in the locator, and the correspondence of the text is done by an internal galen check. Regarding the functional test, the completeness of test coverage has not changed, but due to the fact that the checks are carried out within one test, we significantly save time.

Galen test code.

Our automated system testing uses: Java, Maven, TestNG, Selenium WebDriver, Selenium Grid, Galen Framework.

The cross-platform set of ImageMagick utilities actively helps us in creating Screenshot-based tests.

I would like to immediately note that we write the test code in Java using the PageObject pattern and the Yandex framework - HTML Elements. Maven and testNG are used to run tests.

To make it easier to run tests, view the history of test runs, view reports without involving highly qualified specialists, we are developing separate application- Dashboard.

It would be worth emphasizing that now we are still at the research stage of how to properly organize the entire testing process, and not all approaches have yet been fully mastered and studied

Galen Framework has many undoubted advantages: it is a flexible, easy-to-use tool with extensive responsive design testing capabilities. In addition, it is well documented and is actively being developed at the moment.

The Galen Framework has already been described in some detail in one of the articles. To briefly describe the principle of working with Galen, it looks something like this: you write a page specification (the so-called spec file) using a special, well-documented and intuitive syntax. The spec file describes the relative position, size, indents, nesting of page elements and some other parameters and conditions that the page layout must meet; you can even check the consistency of the text inside the element. And all these checks will be applied depending on the tags we specify.

Tags in the spec file can be set like this:






Galen performs all checks and then generates a visual report in the form of an html file. The report indicates which specific checks failed for a given test, and for each of the failed checks, you can see a full screenshot of the tested page, where the elements that failed a specific check will be highlighted.
For example, a failed test for the distance between adjacent elements will look like this in the report:




When you click on a check highlighted in red, a screenshot of the entire page being checked is displayed with the elements highlighted as follows:




Galen Framework accepts the following parameters as input:

• browser in which the check will take place
• resolution at which to run the test
• url of the tested page
• A Javascript file that needs (if necessary) to be applied on the launched page before starting checks against the .spec file (for example, if you need to check the display of the page for a user logged in on the site)
• name of the .spec file to be launched
• tags that need to be applied to the checks of the .spec file (for example: desktop, all, if we are testing the layout for a desktop).

As you can see, by varying the parameters supplied to Galen, you can achieve almost complete test coverage of our site’s framework.

After we decided on a tool for testing the site framework, the next task was to choose a scheme that would ensure maximum page coverage with Galen tests.

Which pages to choose for testing the layout if the test is intended to test many similar pages?

We decided, without bothering too much, to select a random page from a subset each time we run the test suite (i.e., to test a subset of recipe pages, we selected one of the recipes and passed its url to all layout tests). Since it is not worth checking all the pages of the task, therefore the option of choosing a random page seemed optimal. The URL of a random page of a subset of the pages being tested is transmitted to Galen using a method common to all tests within our automated website testing system (except for layout testing tests, we also have functional and screenshot tests).

For example, there are 2 options for displaying the same type of pages - pages of culinary recipes, in one of which the layout contains an error:




From the example you can see that the “Most Read” block, which should be located in the right column of the page, on the left page is located in the main part, and not on the right. To check that there are no such problems, you need to check big number pages and take into account many factors.

First, the idea came to select the most common devices and use their resolutions to run tests. However, the clearly visible trend of accelerated mobilization of the planet does not allow us to identify (and, even more so, predict) any undisputed leaders in this field. There are a great many devices that allow you to view web applications, and unifying permissions for such devices is completely unfashionable today. The sudden thought that adaptive design That’s why it’s adaptive, so that it can be displayed correctly at any valid resolution, saved our minds and stopped further research in this area. The decision was made: we test the layout on all valid resolutions.

Valid permissions were assigned to all permissions from min Viewport width = 241 px (the browser does not shrink smaller) to max Viewport width = 1920px (the upper limit is a simple force of will). We have not yet had pages where the height of the viewport for the purposes of automated testing was the determining parameter, so we are not paying attention to the height yet.

How to test layout at all resolutions?

To begin with, the entire range of valid resolutions was divided into ranges of different layouts. The layouts themselves are “rubber”, but the different arrangement of the blocks makes it possible to make a distinction. It’s not difficult to determine the boundaries of layouts - we pull the corner of the browser and look at what boundary point the page blocks change: their relative position, number and/or behavior. For simplicity, we take into account only the width of the viewport. The result is the following table:

DESKTOP: max 1920px, min 1018px;
LAPTOP: max 1017px, min 769px;
TABLET: max 768px, min 481px;
MOBILE: max 480px, min 361px;
SMALL_MOBILE: max 360px, min 280px.

By the way, we have decided not to test the SMALL_MOBILE layout yet, since the number of users viewing the Washington Post on devices with such resolutions is catastrophically small (a speculative conclusion, and there is no problem adding it during testing in the future). There are 4 ranges left for testing, with different layouts.

Below is the code to run the Galen test for desktop resolutions:

Hidden text

When running each test, Galen is given a random resolution from the range for a given layout (getRandomResolution(DESKTOP)):

And, in fact, the resolution range is set as follows:

Testing by randomly selecting a resolution from a valid range and a test page from a subset of similar pages thus turns into a probabilistic process. The more often we run it, the more different bugs we will find. With a single successful test, we can only say that this particular page at this particular resolution is valid. But after 500 successful runs, we can say that the layout is mostly viable. Let’s say right away that “500 successful runs” is a speculative estimate, and here you need to look at the content and the number of equivalent pages.

Running at a random resolution very quickly paid off and immediately revealed one interesting bug that we most likely would have missed when running tests at a fixed resolution.

Let's look at how this approach helps us using the example of testing a recipe page.

The recipe page wireframe test runs for a resolution range (Viewport width) from 768px to 1017px. Take for example the page: www.washingtonpost.com/pb/recipes/maple-banana-frozen-yogurt/14143/

The test at the boundary points of the Laptop layout (1017px and 768px) did not produce errors.

However, after we started running the test at a random resolution, in about half the cases the tests crashed and screenshots showed that blocks from the right column were crawling down under the main content.

Correct view:

Carefully! Big picture

Hidden text

The layout is broken:

Carefully! Big picture

Hidden text

Inspired by the article, we decided to use the screenshot-based testing method. By the way, to test the layout, we initially relied on this method. Those. the idea was to compare full-size screenshots of the page with a pre-prepared model, replacing all potentially changing elements with stubs (a pre-selected arbitrary image is taken as a stub). These elements included pictures, flash advertising and text. The idea failed mainly due to the fact that the pages contained many blocks that were loaded dynamically, as a result of which the physical sizes of the screenshots taken and the location of the blocks changed from test run to run. In addition, for some time now Chrome has lost the ability to take full-size screenshots, which also created a number of problems.

Screenshot-based tests now check those individual elements and blocks on the page for which display is important, and/or verification of which with functional or functional tests is difficult or impossible.

For example:

This is what the MostRead block looks like: home page washingtonpost.com (left) and the model with which we will compare a screenshot of this block (right):



The test code looks like this:

The following directory structure is used to store screenshots: /models/HomePage/firefox/HomePage_thePostMost.png

As can be seen from here, for different browsers Take your own model screenshot of the required block.

The shootAndVerify() method finds the path to the model based on the class of the passed page and the name of the browser in which the test is running.

Looking ahead, let's say that it works quite well, and then we will describe some details of the process with the caveat that not everything has been fully debugged.

As it turns out, the snapshot taken of the required block may depend on many factors, such as:

• operating system version
• operating system theme
• browser and its version
• Various font smoothing options and hardware acceleration.

The first problem was that the sizes of the screenshots taken differed depending on the OS and browser settings. To make the block sizes, and therefore the screenshots, the same, you need to launch the browser with constant sizes. You can change the size of the browser window using the corresponding web driver method: driver.manage().window().setSize(requiredSize). But in this way we set the size of the window, and not the size of the visible area we need - the viewport. The vertical scrollbar, by the way, also affects the size of the viewport, and its thickness also depends on windows themes, so you need to take this into account. The solution to the problem was a calibration method that adjusts the size of the viewport to given dimensions. After running the first test, the difference between the width of the window size and the width of the viewport is saved in a special parameter and reused in subsequent runs.

The second problem we encountered was the different display of fonts in browsers due to anti-aliasing settings. We tried to solve the problem by installing various settings browser, such as:

layers.acceleration.disabled
gfx.font_rendering.cleartype_params.rendering_mode
gfx.direct2d.disabled

But, unfortunately, this did not help.

In addition, to compare screenshots, the ImageMagick utility uses a parameter such as fuzz, which sets the maximum possible discrepancy between screenshots.

Tried to solve this problem, experimenting with this parameter. A small fuzz coefficient did not solve the problem, since the number of different pixels was very large due to the fact that there was a lot of text, and a large coefficient led to the fact that the absence of some elements in the blocks did not affect the test, and led to potentially missed bugs.

The solution was to duplicate all the settings of different browsers for everything virtual machines, on which the tests were run, and duplication of the operating system settings themselves.

For example, a test that checks a block of social buttons in which one of the images did not load.

The following links are available in the report:

picture-model


screenshot of the unit being tested:


The result of comparing these two images:


CommandException tells us that the compared images differ by 251px:



There are also situations when the screenshot sizes do not match. In this case, we will receive the following report:




Sometimes, for unknown reasons, elements within the block being tested are slightly misaligned. For such cases, we compare not with one model, but with a group of models that match the mask, i.e. we can have several models of thePostMost block with the following names HomePage_thePostMost.png, HomePage_thePostMost (1).png, and we consider all models valid. Fortunately, the number of such options is finite, usually no more than 2.

As mentioned above, a technology stack is used to write and run tests: Java, Maven, TestNG, Selenium, Galen Framework. In addition, test results are sent to graphite. Tests are launched directly using Jenkins CIS. We will not go into detail about why this particular set was chosen. Let us briefly describe how this is all interconnected.

Selenium Grid is currently deployed locally on four virtual machines with Windows 7, where the grid nodes are running, and on a Linux machine, on which the hub is running. Each node has 3 firefox and chrome browser instances. In addition, Jenkins and graphite are also deployed on the Linux machine. Galen tests run in the general test run thanks to integration with TestNG. For this purpose, a corresponding class was written that allows you to use the jav Galen API. When implementing the interaction of TestNG with Galen, we encountered some problems that were promptly resolved thanks to interaction with the Galen developer. The Galena developer himself is willing to cooperate and regularly releases updates for this tool that expand its capabilities and make it even more convenient. He himself plans to write documentation for integrating Galen with TestNG.

Functional, galenic and screenshot based tests are separated using the corresponding group parameter assigned to the Test annotation, and it is possible to run them separately.

Both approaches - the method of comparing screenshots and testing using the Galen Framework - are applicable for testing page layout. They successfully complement each other. The screenshot comparison method is more applicable when you need to test the display of some individual element or a block, for example a sharing panel in in social networks or the main menu in the header. A block can contain many icons within itself, which in turn can be located inside other icons and elements, or have relative positioning with them.

Using Galen, describing all these small moments is quite labor-intensive, but one screenshot for each browser solves this problem, and comparing screenshots eliminates the possibility that we might miss something when describing a spec. Galen, in turn, does an excellent job of relative placement of blocks and checking the headings and fixed text within them. He has good use when you need to test the layout on the same type of templated pages that are not loaded with functional logic, for example, some information portal, as in our case, when almost any page of the site is accessible without authorization or any other user actions. In addition, Galen is good at solving problems of cross-browser testing in the context of adaptive application layout.

Hello, dear blog readers. It’s not surprising that adaptive design is becoming more and more popular on the Russian Internet. And of course, layout designers need to study it. Since responsive design will soon be on almost all websites, because more and more people are using mobile devices.

And I would like to say that sites with it are much more convenient to read on such devices than without it.

Today I want to introduce you to 5 very useful and cool services with which you can check your website for adaptability.

And so, let's go.

A good service for checking sites. There are many screen resolutions for both tablets and phones.

A simple service for checking a website from Matt Kersley. All popular mobile device resolutions are also available.

A very cool service that will check any site. I really liked the design, as well as the functionality.

Very beautiful and functional service. It is even possible to check how the site will look on TV :-)

Technologies are constantly improving, smartphones and tablets are becoming cooler, and users are becoming more mobile. At the moment, the number of audiences from mobile devices has exceeded users from personal computers. It’s not for nothing that this has led to a whole trend of sites under mobile devices.

For example, if you open a regular website (without adaptive layout and mobile version) through a smartphone, then most likely it will open in a form that is inconvenient for the user. Those. the content will be displayed in full, but there will be horizontal scrolling, which is bad.

Adaptive website layout is like this html layout, in which, depending on the size of the browser window, the site is “transformed” into a user-friendly view

Don't be confused mobile version website and adaptive website layout. The mobile version is located on a separate subdomain and completely duplicates the site content. A responsive site contains the same page URLs, but loads depending on the device different styles CSS, which allows you to display the site in a more convenient way.

Search engines currently have two different results (they are very similar). One for PC users, the other for mobile. If the site is not optimized for mobile devices, then this is considered a negative factor and the site’s position will be lowered by 2-3 (sometimes more). At the same time, it would be logical if the positions of only the mobile version were lowered, but practice also shows that the positions of the main version of the site also deteriorate.

This inequity can be explained by the fact that search engines are more likely to aggregate behavioral factors from desktop and mobile devices. As a result, naturally, if the site is not adapted, then its PF will be worse and this will drag down the main version of the site.

You can check a site for adaptability only by opening it in different browsers and changing the screen width. But do it from a variety of various devices With different resolutions- will take an extremely long time. It’s also unlikely that you have dozens of smartphones and tablets.

There is a much faster and easier solution on the Internet. For example, you can install it in your browser Google Chrome special Window Resizer plugin and use it to open the site in the most popular resolutions.

You can simply manually resize the browser window in width and see the result. Firefox or Google Chrome has a responsive browser design by pressing Ctrl+Shift+M.

The most important condition is to ensure that there is no horizontal scrolling and no flash plugins on the page.

Google was the first to introduce the adaptability factor into its search algorithm. He has a special free service, which analyzes any website for optimization for mobile devices. Yandex introduced this functionality a little later.

After checking, there are two options. Either the site is optimized or it is not:

For example, Google's responsiveness test:

Checking adaptability in Yandex:

Only a person who understands CSS and html can make an adaptive website layout. We will look at the main points, since there is no single recipe.

To make adaptive layout, you need to create CSS style sheets in the third version. There is a difference between 2 and 3, but in this matter the lack of absolute values ​​in styles is very important. In short, all block size values ​​are length, width, dimensions - all of this is specified as a percentage.

CSS @Media Syntax

For example, let's write the conditions under which styles will work for devices with a screen width of less than 800px.

Styles should be written sequentially from high resolution to small, that is, first general styles, and then for “trimmed” sizes, for example:

In the header tags you must include the following line:

The viewport meta tag says that the screen width is the width of the browser, and each pixel corresponds to one pixel on the device. If this is not specified, then adaptability will not be implemented.

For example, there will be a very long word on the page, and then if the overflow property is not set, this may lead to the appearance of horizontal scrolling. To avoid this, you need to write the following to your content: CSS properties

The site sidebar typically occupies a width of around 200-300 pixels, which takes up almost the entire width of the browser on mobile devices. Therefore, drop-down menus are most often created using a standard button in the form of three strokes (this has already become a classic).

You can implement this on the site, but you will have to tinker a little with the styles. Let's look at everything step by step.

The situation when we have a menu and the main content (I did not draw the header and footer):

The HTML code for such a structure could be something like this:

• Link in menu 1
• Link in menu 2
• Link in menu 3
• Link in menu 4

Page Content

Page Content

Page Content

Page Content

Converts on page to

The disadvantage of this method is that at small resolutions the menu will occupy almost the entire width of the area, which means horizontal scrolling will appear, or even worse - all the blocks may float.

Let's modify our example as follows. If the screen resolution is less than 800 pixels, the menu will disappear and a special open menu button will appear.

Here is the html code for the adaptive layout with comments:

• Link in menu 1
• Link in menu 2
• Link in menu 3
• Link in menu 4

Page Content

Page Content

Page Content

Page Content

Let's reduce the screen width to 700 pixels (for example). This is what it looks like on the page

The test shows what a responsive website looks like on various mobile devices. To be tested, your site must support loading and running in frames. For easier viewing, please switch your browser to full screen mode.

Check

A responsive website automatically adjusts to the width of the device. In this case, site blocks can be hidden or modified. For example, on a computer, the site’s logo and menu are located from left to right, and on a phone, from top to bottom.

If the site has a mobile version, then when loading such a site from mobile phone, you will be redirected to another address: site.ru → m.site.ru. The mobile version is a separate site with a different address.

If the site does not have a mobile version, the site address will not change when loading from a phone. The same data will be downloaded to the phone as to the computer. Most often this is inconvenient for the user, because the phone screen is several times smaller.

This problem can be solved in two ways: add a mobile version of m.site.ru or NOT make a separate site, but add it to your main site adaptability. These are special styles and scripts that are turned on if the screen width is too small: for example, hide the menu, increase the font, show small ones instead of large images, etc.

Strictly speaking, you cannot compare a responsive website and a mobile version of a website. Essentially a responsive website = full version+ version for tablets + version for phones (mobile), and all this in one bottle. That is, one concept is contained in another.

An adaptive website combines both a regular website (for PCs), and a mobile one (for phones), and several intermediate options (large phones, tablets, TVs, etc.). The main advantage of a responsive website is that it looks good on any screen width.

The actual number of pixels on modern gadgets is usually very large, and websites are not designed for such a huge width. Therefore, mobile devices with high-definition screens open websites, bringing them to a certain virtual standard. To find out these numbers for yourself, click the button by opening this page from your phone or tablet:

What is my browser window size?

On devices with retina-like displays, the dimensions shown will differ from the actual pixel resolution stated in the device specification. Smartphones will show 320x480 or 480x800 pixels, tablets - 1280x800.

From the author: you have files with layout in your hands. But how can you check its quality? What tools are there for those who don't understand HTML and CSS? What useful things should the layout designer himself use? This article will try to answer these questions.

A question that worries both customers and layout designers. The former can thus make sure that the performer has done his job efficiently. It would be useful for the layout designer to check the site layout before sending it for the client’s approval. This will save time, otherwise a situation may arise when changes will have to be made after the project has been delivered. If you create your own website, verification tools will also come in handy.

Debugger. One of the most simple ways, how you can check the layout of the site - turn on the debugger. It runs when you press F12 in the browser. This tool helps you see how the appearance of a page will change if you remove some elements or styles from it. For example, you added some new CSS property, then added several more. We decided to see how it all would look in the browser.

It turned out that the elements were not displayed as planned. What property caused this error? Determining this manually takes a long time. In the debugger, you can quickly disable any styles and quickly detect errors. It also clearly shows typos. Most experienced coders look for errors using a debugger, rather than looking through the code themselves.

Rice. 1. A debugger makes it easier to find errors.

W3C Validator. Service for checking code for minor errors. W3C is an organization that develops and official support web standards. Because she sets these standards, her website has special service, which can check any page on the Internet for validity (that is, for errors). It must be said that this is a fairly strict validator.

It is often used by customers trying to determine the quality of the layout. Even in a well-designed page, a validator can find more than thirty errors. However, there is nothing wrong with that. The service considers it an error even if you did not put a space before the value of the html attribute. Now imagine that you wrote all the code in this style. You will have hundreds of errors, but in fact the layout will be done correctly, you will just break the standards that the W3C has set for proper coding.

More information about these rules can be found on the W3C website. In fact, the only site that the validator checks without errors is the W3C site itself. Therefore, it is not necessary to correct all errors. Still, the validator may indicate some serious problems, so checking the site layout with it is advisable. validator.w3.org - validator.

IE Tester. There is a program that allows you to test a site in older versions Internet Explorer. Many customers today still require support for this browser so that the site is displayed in it the same way as in others. Perhaps there are already online services where you can perform a similar check. In any case, you just need to paste the required code and the program will show how all this would be displayed in IE 6, 7 and 8.

Usually, no one needs support for the sixth version, and the eighth can behave quite adequately unless, of course, there are new CSS properties in your layout. Older versions of IE have an interesting feature - they read commented code. Therefore, in one of the comments you can include a style sheet that will be created specifically for older versions of this browser.

Other browsers will simply skip this fragment. This method can be used if you really really need IE support.

Other services. Recently, there are more and more services for checking layout. There is no point in dwelling on any particular one. Most of these services work well, but still do not check everything thoroughly.

We should also talk about adaptive layout. Your best tool for this is a regular browser window. Just reduce the width of the window and watch how the design changes. If you see that a horizontal scroll appears or some elements do not fit on the page, it means that your layout is far from ideal and needs to be improved. If you want to see how the site will behave on big screen– zoom out. Of course, if you have a tablet and a phone, you can upload your layout to some free hosting and access the site from these devices.

Rice. 2. Changing the width of the browser window is the easiest way to test responsiveness.

How else can you check the website layout? There must be a check for cross-browser compatibility. You need to open sites in different browsers and see how they look there. You can change the font scale and size. You need to make sure that at least the majority of users will be fine. If you check manually, you can also ask your friends to check the layout in their browsers if their versions are different from yours. Here we can give one more piece of advice - do not use too specific properties that are supported only in one browser. And if you already use them, come up with some alternative for them in other browsers. Some properties still need to use vendor prefixes. This is because web standards are constantly evolving and not all browsers can support everything. But if you already check for cross-browser compatibility, then do it at least for the following browsers: Chrome, Mozilla, Opera, Safari, IE.

There are very good foreign services for advanced cross-browser verification. For example, browserstack.com. This check may cost you money. If you really have a very large project and you want to thoroughly check it, then it makes sense to use such services.

You can also check the layout this way: a screenshot of the layout of the site must be added to the layout in Photoshop as a new layer. Set this layer to transparency and see how well the display of elements matches the layout. You can use a special plugin for this. For example, Pixel Perfect for Mozilla.

Naturally, in any normal layout the encoding and doctype must be specified. You can also check the page for functionality with images turned off or javascript code blocked. The fact is that in the settings of any browser the user can disable these options. It will be especially useful to specify alt attributes for pictures, so that when they disappear the user can at least somehow navigate. In fact, there can be a lot of requirements for layout. Even in RuNet there is a fairly strict list of necessary checks. For example, with the advent of HTML5, layout can be checked for correct semantic markup. However, all this is not critical. The checks described above are quite enough to safely launch a normal, functioning website.

In this article, we described the main ways to check layout using various programs, services and tools. This will help you adapt the site to any conditions.