Github has a lot of integration tools to improve the quality and confidence of the generated code:
https://github.com/integrations/feature/code
For example Travis CI is an excellent service to check it:
https://github.com/integrations/travis-ci
And the best is that the pricing for open source projects is free.
What do you think about it?
Great idea.
I’m familiar with Travis CI, we use it to check and build the User Guide site and Blog using Jekyll.
How could we setup Travis for Emoncms? What sort of automated test do you think would be suitable for Emoncms ?
The basic tests would be check the syntax correction and search missing or unused vars.
For example, the last day I created many “pull request” in dashboard module (you know what I am talking about), one of them with some mistakes. The main mistake was the use of an undeclared var. I suppose that you are checking all the code manually and it is normal merge small changes without testing it. It is impossible check ALL the code.
I think that if we use Travis CI, the develop of the platform will be more agile because the checking of the syntax and other derived problems will covered by Travis CI.
Other desirable target (when the previous topic is solved) is generate good defined cases of use to test the behaviour of the platform. I have for example, problem with the virtual feeds, see the following issue:
In the old forum, I am not the only person with this problem:
https://openenergymonitor.org/emon/node/10977
This problem is an example of the need of tests to check the functionality
Implementing the tests thinking in the desired behaviour, we can achive robust and reliable develop thanks to TDD.
Sounds like a great.
I have enabled Travis CI for the repo and added a minimal php .travs.yml file to the repo:
language: php
php:
- '5.4'
- '5.5'
- '5.6'
- '7.0'
- hhvm
- nightly
A Travis build is attempted but fails with ‘no environment variables set’:
https://travis-ci.org/emoncms/emoncms
What else is required to get Travis to do a basic syntax and var check?
With regard to Virtual Feeds, I would recomend starting a new thread. Virtual feeds were added by @nchaveiro in Emoncms V9, neither I, nor @TrystanLea use them.
Your configuration is running phpunit.
As you don’t have a phpunit.xml (or something like that) in your repository, there is basically nothing to run for Travis. This is the reason beacause you are getting the error in the building.
Take a look to this post:
Search string: “travis The command “phpunit” exited with 2”
I never have performed tests using “phpunit” and I can not help you a lot. I worked time ago with Travis CI and C++ tests. All unit test frameworks are similar and they are “easy” and “fast” to use.
I will investigate about the topic when I have free time.
I’ve done some Googling and can’t seem to find a simple phpunuit example to check php syntax. Maybe it’s more involved than we thought.
Has anyone else experience setting up Travis CI? @nchaveiro @JumpMaster
I suspect that message is just information, rather than an error, and possibly a red-herring.
I’ve just changed the emoncms .travis.yml file to check just one file against phpunit instead of the whole repo and got a green light, passing the Travis checks.
Could the errors be due to a dependency issue?
Does this take us any further forward?
Paul
Correct, good point. The builds are ‘passing’, it’s just a warning. As a test we could commit some PHP with invalid syntax? Can you think of a good example of invalid syntax we could use to test?
Rather than break a core file, I’ve added test.php to the repo, which has an incorrect assertion and should fail.
The incorrect assertion is $this->assertEquals(1+1,1);
If you want to correct the assertion @glyn.hudson hopefully it will then pass OK.
$this->assertEquals(1+1, 2);
Paul
Edit - yes it fails. Travis log extract added below clearly indicating what the problem is.
Ah great. Yes, that works. Build now passes after correction. So is Travis only testing test.php currently as this is the only file mentioned in .travis.yml?
script: phpunit test.php
Would it be possible to get Travis to evaluate all files in the repo for syntax errors?
Good advance!
I think that phpunit is very powerful for core function testing and TDD.
I have investigated a little bit and I found this:
We can try with this for syntax error checking:
before_script:
- if find . -name "*.php" ! -path "./vendor/*" -exec php -l {} 2>&1 \; | grep "syntax error, unexpected"; then exit 1; fi
In the example php lint analyzer is used. This tool analize the php files without execution (static way). This kind of analysis is useful for invalid syntax checking or detect the use of non declared vars.
There are more static analyzers of more higher level like:
But I think that they are out of our scope in this moment. They are for checkings against defined coding standards.
More info: http://stackoverflow.com/questions/378959/is-there-a-static-code-analyzer-like-lint-for-php-files
Yeah, nice this test seems to be working. I added the following to .travis.yml
before_script:
- if find . -name "*.php" ! -path "./vendor/*" -exec php -l {} 2>&1 \; | grep "syntax error, unexpected"; then exit 1; fi
Then committed test2.php with a syntax error echoo instead of echo
The build failed as expected with the error:
$ if find . -name "*.php" ! -path "./vendor/*" -exec php -l {} 2>&1 \; | grep "syntax error, unexpected"; then exit 1; fi
PHP Parse error: syntax error, unexpected '"test"' (T_CONSTANT_ENCAPSED_STRING) in ./test2.php on line 4
Parse error: syntax error, unexpected '"test"' (T_CONSTANT_ENCAPSED_STRING) in ./test2.php on line 4
Which is factly the same error that I got when I ran the incorrect syntax php locally. After commiting a fix the build now passes 
I’ve added a build status icon to the main Emoncms Readme.
Perfect! 
Now the next challenge is create a collection of unit tests to ensure the good working of the core.
Later it is possible to check the behaviour with Selenium. But We must go step by step.