[Omake] Compiling files in different directories

[Aleksey Nogin]

Hugo,

I think one distinction that you need to make is separating the notion 
of "how to build each file _if_ it is needed" from the notion of 
"_which_ files to build". Note that the "how" part does not depend on 
which subproject you are building, but the "which" part obviously 
depends a lot.

Note that OMake also makes this distinction. If you run omake in the 
subdirectory of a big project, the "which" depends on the subdirectory 
you run in, while the "how" part (especially for the targets, such as 
*.cmo, that are specified via implicit rules) depends on the .SUBDIRS 
for the directory where the target resides.

Of course, the "which" and "how" notions are often quite related, but 
they are still two separate notions.

On 31.05.2007 01:20, Hugo Ferreira wrote:

> Aleksey Nogin wrote:
>> [...] doing things like using .SUBDIRS directives
>> with a body (the body then is used in place of the subdir's OMakefile).
> 
> I think this solution may not be practical because the sub-project can
> be quite large and very different in terms of sources so a OMakefile may
> be the better.

Note that you can still split "how" and "which". For example, specify 
"how" in the Common.om file in each directory. Then do "include Common" 
in the OMakefile (which specified the "which") and do

.SUBDIRS: ../xxx
    include Common

in other places. Alternatively put only the "how" part in the OMakefile 
and put the "which" in the OMakeroot.

> Ok. Lets see if I have this right. Assume I have the following in my
> resolve OMakefile (slight change in example):
> 
> ....................................................................
> OCAMLINCLUDES += ../res_stck ../qalist ../term ../symbl_tbl ../compile\
> 		 ../parser-lexer ../unify ../camlp4 ../db
> .SUBDIRS: ../camlp4

Are you sure the above order is the one you want? Note that this would 
cause the camlp4 to inherit the OCAMLINCLUDES variable!

> # OCaml libraries
> OCAML_LIBS = ../qalist/libqalist1 ../res_stck/libres_stck\
> 	     ../term/libterm2 ../symbl_tbl/libsymbl_tbl3\
> 	     ../symbl_tbl/liblexeme2 ../compile/libcompile\
> 	     ../parser-lexer/libllexer ../parser-lexer/liblparser\
> 	     ../unify/libunify ../camlp4/liblogic
> 
> ....................................................................
> 
> Note that ../camlp4/liblogic depends on all other libraries. Note also
> that several of those libraries depend on each other. For example
> ../qalist/libqalist1 is independent but ../term/libterm2 depends on it.
> ../symbl_tbl/libsymbl_tbl3 on the other hand depends on
> ../term/libterm2, which in its turn depends on ../term/libterm2 as
> stated above.

OK, after I see all that I _really_ think that you ought to have a 
single project.

Just see how nicely it would look in a common OMakefile at the top level:

# qalist is independent
.SUBDIRS: qalist

# rest depends on libqalist1
OCAMLINCLUDES += $(dir qalist)
OCAML_LIBS += $(file qalist/libqalist1)

.SUBDIRS: term

# rest depends on libterm2
OCAMLINCLUDES += $(dir term)
OCAML_LIBS += $(file term/libterm)

.SUBDIRS: symbl_tbl parser-lexer unify

...

>> Also, if the camlp4 project already defines the common targets and the
>> resolv project includes the camlp4 directory, why not just drop a
>> separate definition of the common targets from the resolv project?
>>
> 
> I don't know if this makes sense but in each sub-project I have four
> targets:
> * a 'default' test application,
> * launch 'debug' of the test application,
> * 'clean' project
> * a 'library' version
> 
> In the current case I have:
> 1. "/resolv" that requires "liblogic" (library version of /camlp4)
> 2. "/resolv" that requires "libdb" (library version of /db)
> 3. "liblogic" uses a set of libraries call it A
> 4. "libdb" uses a set of libraries call it B
> 5. Some libraries in A and B are the same
> 
> To have "liblogic" I need ".SUBDIRS: ../camlp4"
> To have "libdb" I need ".SUBDIRS: ../db"

So? The whole point of OMake is that it's good at figuring out which 
files need to be [re]built. You can still define one huge project, where 
all the targets you _may_ conceivably need are included. Also may sure 
that you define the appropriate .PHONY targets in the top-level 
OMakefile before any .SUBDIRS. Now, when you run "omake debug" in a 
subproject directory, if will do a debug _for that subproject_ only, but 
if you, say, run "omake clean" from the top-level directory, it will 
clean all the subprojects. (If you need more information, see the 
section of the documentation on running omake from a subdirectory and on 
the hierarchy of PHONY targets - 
http://omake.metaprl.org/omake-rules.html#section:running-from-subdir
)

> My real objective is
> to have "independent" projects in a sense that I can pick and choose
> source files. So in the "resolv" project I simply required:
> 
> FILES = ../db/imap ../db/db resolv
> PROGRAM = test
> OCamlProgram($(PROGRAM), $(FILES))
> .DEFAULT: $(PROGRAM).run $(PROGRAM).opt

Note: you do not need independent projects, independent "subprojects" 
will usually do the trick (and OMake is specifically designed to make 
sure that subprojects stay as independent as possible and that the 
changes you make in one subproject only affect other subproject when 
there is a meaningful dependency).

So, if the resolv is itself a .SUBDIRS in a bigger project that also 
includes db, then the above would work. If you want it to be truly 
separate, you can write something like

.SUBDIRS: ../db
    DB_FILES = $(file db imap)
    export

.DEFAULT: $(OCamlProgram test, $(DB_FILES) resolv)

Note - it does not matter that you defined the DB_FILES variable within 
the .SUBDIRS body (although IMHO it looks slightly better), all that 
matters is that .SUBDIRS has a body, so the ../db/OMakefile (which may 
have lots of "irrelevant" stuff is not included).

Here is what happens here:
  - The .SUBDIRS section estabilishes a "default environment" that says 
_how_ for building _any_ files you might need in the ../db directory
  - The OCamlProgram call establishes _how_ to build the test program 
(_if_ you need it)
  - The OCamlProgram function returns the list of targets if have 
defined (the list will depend on the values of NATIVE_ENABLED and 
BYTE_ENABLED)
  - The ".DEFAULT: ..." dependency specifies _which_ targets to build 
when the omake is called without any explicit target.

To demonstrate the significance of the first note above, here is a 
variation on the above OMake code:

# both db and current dir depend on foo

OCAMLINCLUDES += $(dir foo)

.SUBDIRS: ../db
    # db also depends on bar
    OCAMLINCLUDES += bar

# current dir also needs baz
OCAMLINCUDES += baz

.PHONY: test_progs debug

test_progs: $(OCamlProgram test, ../db/imap ../db/db resolv)

debug: test$(EXE)
    $< some_test_arguments

# By default, build the "best code" test program

.DEFAULT: test$(EXE)

Note that here ../db/imap.cm* will be build with "... -I foo -I bar 
..."; however if you change the order of .SUBDIRS and the top-level 
"OCAMLINCLUDES +=", then a different set of include directories will be 
used. Hopefully this demonstrates the importance of the SUBDIRS directive.

Aleksey
-- 
Aleksey Nogin, Research Scientist
Advanced Technologies Department, Information & System Sciences Lab
HRL Laboratories, LLC, Malibu, CA