Directory structure

[milancurcic]

Mostly cosmetic, but affects quality of life: What kind of directory structure should we use for the library?

Potentially touching on #2 and #3.

I tend to like structures like this:

stdlib/
  src/
    lib/
      mod_stdlib.f90
      subdir1/
      subdir2/
      ...
    tests/

[zbeekman]

Tests

I really think that it's important to make it easy for people to write and add tests, preferably before they write the implementation.

In my ZstdFortranLib project adding tests for a source file that makes up a library or executable target is as simple as adding a corresponding file with the same name, but ending in _test in the test directory. I have some CMake logic that searches through all Fortran sources associated with a target and looks for corresponding test files, and then automatically adds them. In this way, adding tests for a new source file is easier (from the build system perspective) than adding the actual source file is.

.e.g., if foo.F90 is used in libfoo then putting foo_test.F90 in the test subdirectory automatically creates the test executable, links it to libfoo and adds the CMake test.

Flat is better than nested

In general, I think we should strive for a directory structure that is as flat as possible, and only start adding subdirectories when we reach some threshold of 3-4 files implementing a well defined, cohesive idea. I also generally like having a test or tests directory in each source directory, i.e., lib/tests, subdir1/tests, etc too because it narrows the scope of looking for a test that corresponds to a source file. (Or even just putting tests along side with the _test.[Ff]90 ending in the same directory until it gets too cluttered to be sustainable.)

[certik]

I agree with @zbeekman to have tests and more flat structure. I usually name tests as test_*.f90.

[milancurcic]

Great, I agree, I like the flat structure as well. I'd caution against keeping test source files together with module files at first. We don't have any modules now, but where we want to arrive is a place with a ~dozen to a few dozen modules. In a flat directory structure, it'd soon become cluttered to keep test programs in the same directory.

Less important and personal preference, but when I explore libraries, I first like to look in the library source directory and see what does this library have unless already covered in the README.md. Test files would get in the way of readability here. We should think of the first impression that a new user gets when landing onto the repo page.

[certik]

I think @zbeekman is suggesting to keep tests in the tests subdirectory. I think we all agree on that one, so that the tests do not clutter the source files.

[milancurcic]

Yes, I definitely misread that. :) We're on the same page.

[zbeekman]

Yes, I'd recommend putting tests in a test or tests subdirectory of the directory that holds the sources to be tested: E.g., src/lib/subdirA/tests for unit tests and specs corresponding to files in src/lib/subdirA. At least for unit tests and specs this makes sense. For integration tests or any test that doesn't have an obvious one-to-one mapping a test or tests directory at the project root (or directly under src if src/lib is where the library code goes) is needed to hold them.

I'm not sure I see the benefit of src/lib/... over just src/ unless we want to pack, e.g., development scripts, etc. into src/, but I'm open to being convinced/wrong. 😉

[milancurcic]

Great, so if we have tests/ in each sub-directory, then I agree that src/lib is redundant, and go with just src/. Given this, how would you organize tests for modules that don't have subdirectories, for example, if our first implementation module is ascii.f90 from #11, do we have:

src/
  stdlib.f90
  ascii.f90
  tests/
    test_ascii.f90

[zbeekman]

That looks good to me. I typically name the directory `test` without the s (one less letter) and name the files <name>_test.f90 but either convention is fine by me. As stuff gets added if src gets too cluttered you can break out more nesting and structure. E.g. `src/strings/ascii.f90` etc. where other string stuff gets gets grouped there.

…

On Tue, Dec 17, 2019 at 6:29 PM Milan Curcic ***@***.***> wrote: Great, so if we have tests/ in each sub-directory, then I agree that src/lib is redundant, and go with just src/. Given this, how would you organize tests for modules that don't have subdirectories, for example, if our first implementation module is ascii.f90 from #11 <#11>, do we have: src/ stdlib.f90 ascii.f90 tests/ test_ascii.f90 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7?email_source=notifications&email_token=AACEIPFAQKXOMPFNEQLGGL3QZFOFFA5CNFSM4J25IBJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHEJ5UQ#issuecomment-566795986>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACEIPH6OOVQZXINDO6RZ4TQZFOFFANCNFSM4J25IBJA> .

[certik]

@milancurcic that's precisely how I would do it.

[zbeekman]

Is this finalized? Should I work on CMake helper functions to detect and add tests?

i.e. src/ascii.f90 --> src/tests/test_ascii.f90?

[certik]

Isn't it better to list tests explicitly? I feel the automatic detection requires programming in CMake and I would prefer to avoid as much programming as possible in CMake, because CMake is not a very good language for programming. Also, I think a lot of people asked us to also maintain manual Makefiles. Which is quite easy if we keep CMake programming to minimum.

[zbeekman]

Sure, we can take either approach, but automatic addition of tests is really nice for contributors because THEY don't have to program in CMake... They just create a test and it gets added and run. Otherwise they need to call add_executable add_test target_link_libraries etc etc. With automatic detection creating a file with the right name in the right place causes the test to get added, compiled, linked and run...

[zbeekman]

I personally think this is worth the CMake "headache" and I am happy to maintain and create it...

[milancurcic]

They just create a test and it gets added and run. Otherwise they need to call add_executable add_test target_link_libraries etc etc.

If done in a loop, it's really just typing out the test name, which is minimal work.

I recommend against depending on a "key man" for any specific task, for example Zaak's CMake expertise. If we keep the workflow simple enough, a developer can (and should) work through the whole cycle: Writing code and tests, documentation, and adding to build systems. On one hand, automating things like adding tests to CMake will help every developer. On the other hand, everybody having to "touch" each part of the workflow will help them be more versed in the whole workflow. In a nutshell, growing a community of generalists rather than that of specialists.

The question of maintaining manual Makefiles raises a deeper question: How do we maintain build systems that we agree on? I see the value in manual Makefiles, but we would as a community have to agree that everybody who contributes a feature also has to contribute the changes to all build systems that we agree to maintain. Specifically, we can't have one person running around patching up manual Makefiles while other developers are rapidly adding and updating the API.

Should we discuss maintaining CMake and manual Makefiles in separate issues (they're off-topic here)?