Skip to end of metadata
Go to start of metadata

This document describes the strategies to apply to handle errors in the Sonar ecosystem.

Listing of expected errors

List of topic statuses (TS)

  • [No status]: no discussion has been started so far
  • (lightbulb): a discussion has started on this topic
  • (grey lightbulb): the discussion has been temporarily held on (stand by)
  • (tick): dicussion completed, and guidelines provided

List of behaviour statuses (BS)

  • [No status]: don't know / topic info not completed
  • (error): incorrect current behaviour
  • (tick): correct and expected current behaviour

General

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotesReferences
Use of RuntimeException all over Sonar code base  Currently, we can find places in Sonar where RuntimeException is thrown, and other where SonarException is thrown. We should be consistent with this topic. 

None of them. IllegalStateException and IllegalArgumentException are usually enough.

Generally, using RuntimeException is not good, and it is actually raised as a violation. 

Server

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotesReferences
Invalid environment       
Missing/bad $SONAR_HOME       
Missing Java executable (standalone mode only)       
Unsupported Java version       
Missing configuration files (conf/)       
Not enough space on disk   Is it really required to verify space availability ? Contrary to Jenkins, space needs do not increase through time. Moreover server does not require too much space.   
Unsupported JEE server   

Not verified. What should be done ? Hardcode a list of supported servers and log a warning/crash if server is not in list ?

-> not a big ROI

   
Database is down       
Multiple JAR files in extensions/jdbc-driver/<dialect>(tick)(tick)Clear message in logs then crash    
Invalid temporary directory (not defined or not writable)(tick) ?java.io.tmpdir must be set to $SONAR_HOME/tmp. This directory must be created during startup if it does not exist.  SONAR-3399
Invalid configuration       
Invalid Logback configuration       
Missing required JDBC properties       
Invalid JDBC credentials       
Invalid JDBC URL       
Invalid JDBC driver       
Invalid Oracle connection  Misleading log, original error is lost. Discussion.   SONAR-3259
 General       
 Startup task failure (register metrics, rules, ...)       
Startup failure due to a plugin       
Startup failure because plugin requires a greater version of sonar  Pre-condition is not verifiedLog a warning + ignore this plugin + mark it as disabled in update center   

Web application (UI)

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotesReferences
 Ajax requests       
Error (status != 400)       
Access denied       
Server is down       
 General       
Page not found (unknown controller/action)       
Resource not found       
Missing required parameters       
Bad parameters       
Not logged in       
Not enough permissions       
Internal error       
Database is down       
Database migration failure       
Browse pages during database migration       
Error in widget extensions       
Widget template does not exist      See this discussion
No name or description for rule(tick)(error)NPE (at least with Sonar 2.14)Avoid NULL : make name and description required, and so notify developer as soon as possible by providing exception with a meaningful message  when default name or description wasn't provided.  SONAR-3305
Exception from Java code (e.g. from JRubyFacade)(tick)(tick)Since Sonar 2.15 we have special handler on Ruby side (render_native_exception), which is able to convert Java exceptions into HTTP response with correct status and message (currently IllegalArgumentException => 400 and all other => 500).   SONAR-2424

Web services

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotesReferences
Server-side       
Unknown web service (/api/unknown)       
Unsupported format       
Bad parameters       
Missing required parameters       
Access denied       
Internal error       
Database is down       
Error in web service extensions       
Java client       
Not compatible with server version       
sonar-ws-client throws ConnectionException without good information about error (error)   

Evgeny:  For example there is no way to distinguish :

  • HTTP-transport error ( timeout, proxy authentication required, ... ),
    because it provides only message, but not error code, whereas client
    of API can make proper call in future based on error code, e.g. provide
    credentials for authorization
  • Incorrect usage of web service
  • Internal server error
  • and so on

SONAR-2780

Code inspection

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotesReferences
Configuration       
Wrong JDBC settings (error)Message is correctly logged but is a bit lost in the stacktraceNo stacktrace, only the message   
Wrong version of db schema (error)Picocontainer error with misleading message. It can occur when version of org.codehaus.sonar:sonar-maven-plugin is fixed with a bad value.Clear message that shows de-synchronization between batch and server  SONAR-3292
General       
Partial analysis due to limitations in algorithms     In some cases we are not able to complete analysis for specific files due to limitations in algorithms ( detection of duplications, parsers, ... ). This situation is pretty similar to topic "Lexer and Parse errors/warnings".CPD - SONAR-2904 , Checkstyle - SONAR-3032
Not clear which plugin fails     In case of failure of a component from some plugin (Sensor, Decorator, custom extension) we can guess name of plugin only based on order of execution from log, but not from exception.http://markmail.org/message/welszsv6jvw65xdp
When ClassLoader fails to load plugin, we don't know details     Would be better to know name of problematic class, and name of problematic plugin or at least name of jar file.SONAR-3267
External tool faillures (error)Most plugins (but not all of them) throw a SonarException to fail the analysis, which prints out a full stack-trace

When an external tool fails:

  • The analysis should be stopped
  • The log should only display a meaningful message that contains the error code, but not the full stack-trace (because somehow this is a "functional" error)
  1. CommandExecutor should encourage/promote the check of exit codes (SONAR-2969)
  2. And we should have a specific runtime exception to tell Sonar Batch to stop the analysis without printing the stack-trace but only the message.
 see SONAR-2969
Put error codes/ references in logs     Dinesh/Evgeny: It might be interesting ( in a long term ) to introduce something like error-codes or references to Confluence. To help people quickly understand problem and find additional information, which might be updated on-the-fly even after release, thus to reduce support in ML.
Problem : how to make link permanent.		 
Log output not appropriate place for recoverable errors    
  • fail the build
  • as was mentioned by Dinesh: create violation (in case of parsing error)
  • find a way to deliver warning to Sonar Web interface
Log output not appropriate place for recoverable errors, because in continuous environment if build was not failed, then everything is ok.SONAR-3031, SONAR-3032
Extension point failures    Explicitly wrap such points in try-catch block and provide additional information.Evgeny: Any Sonar extension point is a point of failure, because Sonar will execute non-trusted code, which comes from third-party plugins. If execution failed, then error message might not contain contextual information, like execution of Decorator for Resource. 
Don't pass internal error directly to user     Evgeny: I believe that any internal error (like SQL error) should not be passed directly to user without providing additional information. 
Code inspection during database upgrade       
Configured database does not relate to the configured server     Evgeny: I guess this relates to SONAR-3126 
Better handling, when missing language plugin      SONAR-3125
Corrupted plugin was downloaded from Server       
Corrupted batch libraries were downloaded from Server     This does not include Maven Plugin, because Bootsrapper not used, however it should. 
Analysis fails because of the initialisation of an "unrelated" plugin (error)All available extensions are currently loaded by Pico, whatever the language is.
If a plugin class fails in its constructor (bad practice...), this fails the build even if it's not related to your language.		The guilty plugin should not fail the build as it shouldn't be part of the analysis. Fabrice: already discussed that sometime ago with Simon, this is a tricky problem...See issue on C# Plugins 1.1: one of the extension classes of the 1.1 C# plugin used to do some specific initialisation in its constructor => this class would get instantiated by Pico even though the language is not C#, and in case of a Java project for instance, the constructor would fail, thus stopping the analysis
Deprecated or not existing property     e.g. "sonar.cobertura.maxmen" 
Huge stack-trace and exception wrapping      problem from real life : http://markmail.org/message/hzgxnrdbib26lsms
Incorrect usage of Sonar API or incorrect implementation of Sonar SPI by plugins     Should produce meaningful messages for users. For example instead of just throwing IllegalArgumentException would be better to have message "Bug in plugin ...". 
Decorator failure (error)Decorator name is missing, for example with Maven :

Embedded error: Fail to decorate 'org.apache.commons.dbcp2.BasicDataSource'
Measure value can not be NaN"

Note that enabling the Maven parameter -e fixes the issue.

    
Hard to distinguish : problem with source encoding or duplicated resource      

problem from real life : http://markmail.org/message/i7ne5pyeqvh42weg

Language Plugins

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotes / References
External tools      
Bad path for external tools(tick)  

Analysis failure with a meaningful message, which should include:

  • name of tool
  • path, which was used
  • maybe instructions on how to change path
Analysis failure with message containing the name of the tool together with the incorrect path which was given + Instructions on how to disable the tool execution

Fabrice: I don't agree to fail the analysis all the time. IMO, for external tool failures, I'd like to have a system where I could tell Sonar batch to:

    • stop the build (because the tool is mandatory)
    • continue the build (because the tool is optional). In that case, we should have a mechanism to tell the end user that the analysis was "partial" and to list which tools failed and which data is consequently missing

This "generic" solution is valid for all the cases below also.

Evgeny Mandrikov : Could you please provide an example of optional tool and when partial analysis is acceptable ? I'm asking because any partial analysis affects historical data and so timelines.

    • FB: I was thinking of plugins that produce non-core metrics (for instance, the JIRA or Trac plugins => users might want their quality analysis even if those metrics were not computed?). We could even imagine a admin page to tell which plugins should be considered as mantory or not, and let Sonar batch handle the exceptions accordingly (this way you could set "alpha-version plugins" as "optional" to not fail an analysis). This is just an idea (wink)
    • DB: I think Fabrice that this is a "higher level" issue. If for instance the JIRA plugin cannot connect to JIRA (because of a 404, so it's a "bad path for exeternal tools"), it should fail for sure. Then, the code analysis part in the batch of Sonar can decide whether or not to stop the analysis, but it is definitely outside of the scope of language plugins.
Failure while running external tools(tick)  

Analysis failure with a meaningful message, which should include:

  • name of tool
  • output from tool
  • exit code and expected one in case of mismatch
Analysis failure, with message the name of the tool together with an explanation of the error (in case of exit code, the expected one and the actual one)  + Instructions on how to disable the tool execution.Ex.: bad exit code or error grepped from STDOUT
Resulting file of external tool not found(tick)  

Analysis failure with a meaningful message, which should include:

  • name of tool
Analysis failure with message containing the name of the tool together with the incorrect path to the result file  + Instructions on how to disable the tool execution. 
Parsing error in resulting file of external tool(tick)  

Analysis failure with a meaningful message, which should include:

  • name of tool
  • path to resulting file
Analysis failure with message containing the name of the tool together with the path to the result file and an explanation for the parse error  + Instructions on how to disable the tool execution. 
Timeout(tick)  

Analysis failure with a meaningful message, which should include:

  • name of tool
  • current timeout
  • maybe instructions on how to increase it
Analysis failure with message containing the name of the tool + current timeout value + instructions on how to increase the timeout level  + Instructions on how to disable the tool execution.Important note from Dinesh Bolkensteyn : timeout can be defined in different ways, e.g. total value, function based on "per file" value, "per group" value, ...
General      
Invalid license(tick)(error)Just a little warning in the log, but analysis is successful, without executing the sensors, so we receive an empty analysis for project.

Generally batch must NOT fail when a license is invalid, but language plugins should fail if the project language matches.
Example : C++ plugin must not fail if Java project and invalid license.

TODO decrease duplication and provide an API for this case

 
Lexer and Parse errors/warnings(tick)  Log error, make a mark in file (violation / finding / metric). Try to recover from error, in case of unrecoverable error - skip file.

TODO finding / metric should be discussed with Freddy

 
Bad parameters given to checks(tick)  

Each rule (enabled during current analysis) must validate parameters and if they incorrect, then analysis must fail with a meaningful message.

TODO Long-term: We should think about providing an API for validation in UI on the fly (callback to server). 
   Cross-project Sonar CPD doesn't work, if source files was incorrectly imported by language plugin  http://markmail.org/message/oteilf2ouypthf2l

Java Language Core Plugins

TopicTSBSCurrent behaviourExpected behaviourPossible implementations/solutionsNotesReferences
General       
Class files outdated in comparison with source files (error)Incorrect analysis data will be injected into Sonar DB.Detect and somehow notify user.

Simon Brandhof : For each source file :

  1. does a class file exist with the same name ?
  2. compare last update times with class file

But how to detect that a source file has been removed but its class is still available ?

Evgeny Mandrikov : What prevents us from visiting class files to check that corresponding source file exists ?

Simon : yes, but only for public classes

This is especially important for analysers like FindBugs.
We should find a way to detect such situation.		see http://markmail.org/message/rev66nnjkpah4b56
Missing class files  Incorrect analysis data will be injected into Sonar DB.  This is especially important for analysers like FindBugs. For some rules from Squid, Checkstyle, PMD. 
sonar-squid-java-plugin       
A common misunderstanding of a source root directory     Problem is that exception message does not contain enough information for explaination of problem, at least because directories are listed far before exception.http://markmail.org/message/y7oyjo3jd33efu4c
Labels
  • None

1 Comment

Hide/Show Comments

  1. Jan 27, 2012

    Dinesh Bolkensteyn

    Evgeny,

    I disagree with the sentence you just added, "Incorrect usage of Sonar API or incorrect implementation of Sonar SPI by plugins, should produce meaningful messages for users. For example instead of just throwing IllegalArgumentException would be better to have message "Bug in plugin ...".".

    I believe that an IllegalArgumentException should be thrown with a meaningful message in case of a correct Sonar API usage.

    It is only when Sonar (whatever that means, the "batch" side during analysis but perhaps also the "server" side for other kind of plugins) catches an exception, which was not caught by the plugin itself, that a "Bug in plugin... try to disable it..." kind of message should be displayed.