Continuous integration

Continuous integration (CI) runs tests and checks automatically on every pull request and every push, so that bugs, regressions and lower code quality are caught before a release.

LimeSurvey uses GitHub Actions. The workflow files are in .github/workflows/. How to set up and run the tests on your own computer is described in Regression and unit tests.

Overview

Unless stated otherwise, a workflow runs on pull requests against master, develop-minor and develop-major, and on pushes to these branches. Pushes to other branches are only tested once a pull request exists.

Workflow (file)	Name in GitHub	What it checks
main.yml	LimeSurvey - Backend Unit Tests & Code Quality	PHP unit tests on several PHP versions and databases, code coverage, static analysis, dependency audit. Skipped when only editor/ changed.
functional.yml	LimeSurvey - Functional Tests	Browser tests with Selenium and Firefox against a real Apache installation
frontend.yml	LimeSurvey - Frontend - CI pipeline	Builds the admin JavaScript packages and runs their Jest tests. Skipped when only editor/ changed.
editor.yml	QuestionEditor - CI pipeline	ESLint, Prettier and Jest for the React survey editor in editor/. Runs on every push to any branch.
codespell.yml	Codespell - Spelling Check	Spelling mistakes in code and comments
labeler.yml, review-labeler.yml	Label PR Conflicts, Label PR Review Status	No checks. They manage pull request labels, see Pull request labels.

  The master branch has no required status checks, so GitHub allows merging a pull request with failing checks. Check that all workflows are green before you merge.

Backend unit tests and code quality (main.yml)

Unit tests

The unit tests in tests/unit run with this matrix:

PHP	Database
8.1	MySQL 8.0
8.5	MySQL 8.0
8.5	PostgreSQL 14
8.5	Microsoft SQL Server 2022

Each job installs LimeSurvey with the console installer and runs PHPUnit twice:

1. Normal run with code coverage enabled.
2. Run with LIMESURVEY_DEBUG=2. Debug mode turns PHP warnings and notices into errors, so this run finds warnings that the first run hides.

Rules for tests that follow from this:

• PHPUnit runs with --fail-on-skipped, so a skipped test fails the build. Do not use markTestSkipped().
• Tests that only work on MySQL get the annotation @group mysql. This group is excluded on PostgreSQL and MSSQL.
• Tests that cannot run in debug mode get @group debugoff. This group is excluded from the second run.
• --stop-on-failure is set, so a job only reports the first failing test.

Code coverage

Coverage is measured only for application/models/services (see <coverage> in phpunit.xml). The test-coverage job takes the result of the PHP 8.5/MySQL run and fails if coverage drops below 37 % (tests/bin/check_coverage.php). In October 2026 coverage was about 43 %.

Code check

The code-check job runs on PHP 8.5:

Step	Fails the job?
composer audit: Composer packages with known security vulnerabilities	Yes
composer test: PHP syntax check of application/ and tests/, PHP Mess Detector on selected folders, Psalm with psalm.xml and psalm-strict.xml (see the scripts section of composer.json)	No, the result is only informational
Psalter and PHPCBF auto-fix: fixes some Psalm issues (MissingOverrideAttribute, UnusedVariable, UnnecessaryVarAnnotation) and PSR-12 code style in selected folders	Yes, if anything was fixed
Psalm on the whole application/ folder with low strictness (psalm-all.xml)	Yes

If the auto-fix changes code, the workflow pushes a commit named "Apply PHPCBF and Psalm auto-fix changes" to the branch and then fails. Pull the branch, review that commit, and push again. To avoid this, run PHPCBF and Psalter locally before you push (see below).

The folders checked by PHPCBF are listed in the Run PHPCBF step of main.yml, and the coding standard is tests/rulesets/phpcs_ruleset.xml. Code outside these folders is not checked for code style yet.

Functional tests (functional.yml)

The functional tests run in 13 test suites (survey, question-a, question-b, expression, validation, navigation, import-export, theme, admin, user, plugin, security, api), each on PHP 8.1 and 8.5, so 26 jobs in total. The suites are defined in phpunit.xml.

Each job:

• installs PHP with Apache and MySQL and installs LimeSurvey,
• disables the new survey editor (editorEnabled => false). Only the admin suite builds the editor,
• starts Selenium 3.9 with headless Firefox,
• runs the suite with --stop-on-failure --fail-on-skipped.

When a job fails, screenshots from tests/tmp/screenshots/ are attached to the run as an artifact (screenshots-<PHP version>-<suite>) for 5 days.

  CI runs only the files listed in the 13 suites in phpunit.xml. A new test file in tests/functional/ that you do not add to one of these suites never runs in CI.

Frontend packages (frontend.yml)

Builds the admin packages assets/packages/adminbasics and assets/packages/adminsidepanel with Yarn on Node.js 24, and runs the Jest tests of adminbasics. A newer push to the same branch cancels a running build.

Survey editor (editor.yml)

For the React editor in editor/:

• ESLint with --max-warnings 0: every warning fails the build.
• Prettier format check.
• Jest tests with coverage. These only run when ESLint and Prettier pass.

Spelling (codespell.yml)

Runs codespell. Words that are spelled correctly but flagged go into the ignore list in .codespellrc. Folders to skip must be added both to .codespellrc and to the skip option in codespell.yml, because the GitHub action overrides the skip list from the config file.

Pull request labels

• A new pull request (not a draft) gets the labels Needs code review and Needs testing.
• When a review approves the pull request, Needs code review is replaced by Code review done.
• A pull request with merge conflicts gets the label Has conflicts. The label is checked again on every push to the pull request and to its base branch.

Reproducing failures locally

Set up the test environment first as described in Regression and unit tests. Then run the command matching the failing step from the LimeSurvey root folder:

Failing step	Command
Unit tests	php vendor/bin/phpunit --fail-on-skipped tests/unit
Unit tests in debug mode	LIMESURVEY_DEBUG=2 php vendor/bin/phpunit --fail-on-skipped --exclude-group debugoff tests/unit
Unit tests on PostgreSQL or MSSQL	Add --exclude-group mysql (in debug mode: --exclude-group debugoff,mysql)
Functional test suite	php vendor/bin/phpunit --fail-on-skipped --testsuite <suite name>
Psalm	./vendor/bin/psalm -c psalm-all.xml
Psalm auto-fix	./vendor/bin/psalter --issues=MissingOverrideAttribute,UnusedVariable,UnnecessaryVarAnnotation
PHPCBF	./vendor/bin/phpcbf --standard=tests/rulesets/phpcs_ruleset.xml <folder>
Composer audit	composer audit
Informational checks	composer test
Survey editor	cd editor && npm run eslint && npm run prettier:check && npm test
Spelling	codespell (reads .codespellrc)

CI runs on PHP 8.1 and 8.5. A test can pass on one version and fail on the other, so check which job failed.

Open points

• Two functional tests are not part of any CI suite and never run: tests/functional/frontend/DashboardLoadedWithCorruptedSurveyTest.php and tests/functional/frontend/PrefillStartingValuesTest.php.
• The syntax check, PHP Mess Detector and strict Psalm (composer test) do not fail the build.
• Code style (PHPCBF) and coverage are only checked for some folders.
• No metric for code duplication.
• There are no automated accessibility tests (see Accessibility).

See also

• Regression and unit tests
• Code quality guide
• Standard for Git commit messages