Incident

Even though all checks are run continuously, it is sometimes needed to run a check manually. Leankoala provides four features to start them:

• Rerun a single check
  Every event/incident on the status dashboard provides a rerun button. Clicking it the check will be added to the Leankoala queue and will be processed in most cases within the next seconds. 
  If not all checks are visible it is possible that a dashboard filter is set. 
• Rerun all checks for a given system
  The system overview (main menu) provides a link to rerun all checks for a given system. This can be used after a deployment or when some other major changes were made. As this feature can trigger a lot of checks it is only allowed to run it once every five minutes.
• Change check configuration
  Every time a single check configuration gets changed the check will be rerun automatically. If the change is system-wide the changes will only take effect after the next cyclic run. 
• Deployment hooks
  In many cases, checks should be run after a deployment. Leankoala provides a deployment hook [configuration] to do this automatically as a task within a deployment script. 

The strict mode is the default mode for all our checks. When it is activated a check is considered as incident as soon it occurs. For most web systems that is the best way to handle found anomalies.

But there sometimes also is a need for a less strict error handling. There can be many reasons why not every single incident is an error:

3rd party integrations: there is often a 3rd party integration (like an ad) that is not very stable. That’s mostly not a problem for the end user but would trigger the Leankoala checks.

high availability: not all systems need to be available 24/7. For some components, it’s ok to be unreachable most of the time. So if you can’t ping it that’s ok.

browsers are divas: one of the unique selling points of Leankoala is the usage of real browsers instead of just requesting the servers HTML. But that sometimes comes with a downside. Errors that are seen by a browser can be gone the next time you reload the page.

It does not matter why tests randomly fail they are annoying. That’s why it is possible to switch to the non-strict mode. When this mode is active an incident only gets opened if a check fails two times in a row. This preference applies on a check (combination of tool and system).

Sometimes tests are flaky and it is hard to find the reason why. Some of them are reoccurring and can be found on all kind of websites:

• Advertisement - Ads can be very problematic because they change very often. So if Leankoala opens for example an incident concerning a JavaScript error it might be because of an ad. When the developer checks the URL there is another ad loaded and the error is not triggered.
   
• A/B tests - A unique selling point of Leankoala is the collection wizard. When adding a new URL to monitor Leankoala tries to find all known pattern (like Google Analytics or a simple headline) and adds them to the projects checks. The problem here is that when an A/B test is running the rendered page does not look the same for every user. That means that in one test the A-version of the website was rendered in the other B. Patterns that were found by the wizard don't have to be there anymore.