How to run unit tests?

I have a few ideas for stuff I’d like to contribute. I cloned the github repo and made some changes, but I fail to run the unit tests. See the error below. My setup is simple

  • Installed Python 3.6
  • Cloned the github repo
  • Went to my local github repo and ran pip install -e .

Now when I go to psychopy/psychopy/tests and run python run.py, it opens up two pyglet windows and begins collecting items. Then it fails with a long traceback, ending in:

  File "D:\offline\git-psychopy\psychopy\psychopy\tests\conftest.py", line 14, in pytest_sessionfinish
    pytest.app.quit()
AttributeError: module 'pytest' has no attribute 'app'

pytest is version 4.4.0. Here’s the full output:

================================================= test session starts =================================================
platform win32 -- Python 3.6.0, pytest-4.4.0, py-1.8.0, pluggy-0.9.0
rootdir: D:\offline\git-psychopy\psychopy, inifile: setup.cfg
collecting 748 items / 25 errors / 723 selected                                                                        INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 209, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 248, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 258, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 495, in perform_collect
INTERNALERROR>     self.config.pluginmanager.check_pending()
INTERNALERROR>   File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 251, in check_pending
INTERNALERROR>     % (name, hookimpl.plugin),
INTERNALERROR> pluggy.manager.PluginValidationError: unknown hook 'pytest_namespace' in plugin <module 'psychopy.tests.conftest' from 'D:\\offline\\git-psychopy\\psychopy\\psychopy\\tests\\conftest.py'>
Traceback (most recent call last):
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\jonas\AppData\Local\Programs\Python\Python36\Scripts\py.test.exe\__main__.py", line 9, in <module>
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\config\__init__.py", line 79, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 62, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 242, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 235, in wrap_session
    session=session, exitstatus=session.exitstatus
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\hooks.py", line 289, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 68, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\manager.py", line 62, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 203, in _multicall
    gen.send(outcome)
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\_pytest\terminal.py", line 656, in pytest_sessionfinish
    outcome.get_result()
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\users\jonas\appdata\local\programs\python\python36\lib\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "D:\offline\git-psychopy\psychopy\psychopy\tests\conftest.py", line 14, in pytest_sessionfinish
    pytest.app.quit()
AttributeError: module 'pytest' has no attribute 'app'
3.8945  WARNING         pygame audio lib was requested but not loaded: ModuleNotFoundError("No module named 'pygame'",)
4.0423  WARNING         Import Error: No module named 'psychtoolbox'. Using event module for keyboard component.
4.6242  WARNING         psychopy.parallel has been imported but no parallel port driver found. Install either inpout32, inpoutx64 or dlportio
1 Like

Hi @lindeloev, I like to run specific tests, rather than the whole test suite. For example, if I am working on the JS code compiler, I navigate to tests/test_scripts and run:

pytest name_of_test_file.py

Ah, I think this looks like an issue with changes to pytest. I recall something from Richard that pytest fixtures were changing and maybe this is the result of that.

It used to be that we could simply create a global variable and attach it to some class/lib that wasn’t going away. so creating the app and attaching it to pytest module was an easy solution. It looks like we now need to create a file called conftest.py that configures global fixtures

I think it relates to this, so @lindeloev might need to downgrade his version of pytest to < 4.1.0.

Yes, @dvbridges! Doing pip install pytest==4.0 solved the first problem.

@jon, I see you commited some fixes, so I’ll try with the newest pytest once they are pulled. Great!

Then the tests failed prematurely because I didn’t have vlc so one pip install vlc later, everything ran. I don’t know much about installing, but would it make sense to add vlc to setup.cfg or would that also include it in the standalone?

Further info… yes, this appears to be caused by a deprecated feature being removed in (just after?) pytest 4.0
https://docs.pytest.org/en/latest/deprecations.html#pytest-namespace

We were already using the conftest.py system that I mentioned above, but I’ve had to change the way that is used. Unfortunately we are still having some additional problems on the Travis test suite that I don’t understand yet. It appears that Travis completes the tests fine (reports only deprecation warnings) but then raises an error during shutdown and considers that, overall a fail. :frowning:

I hate test suites!

But the issue specifically around pytest==4.0 should (I think) now be fixed

It looks like you reverted some of the changes. I cannot get the unit tests to run with pytest 4.4 with the same error as @lindeloev. All the demos seem to work in the app. I just tried with 3.0.7 and 3.1.0. Is the solution still to rollback pytest?

Ah, yes, I’m afraid I spoke too soon - I got new pytest to work with Python2.7 but not with Python3.6 and had to revert to the deprecated style until I have longer to look into why it didn’t work