[Omake] OMake locking

[Aleksey Nogin]

On 17.01.2007 11:03, Jared C. Davis wrote:

>> This is not really about locking - the basic assumption of OMake is that
>> it does the global dependency analysis on the whole project, so in
>> general you can not really have more than one instance of OMake running
>> on the same project, without loosing some of the reliability guarantees
>> that OMake is designed to provide. Also, there is a global per-project
>> "build state" database (.omakedb) that is not designed for parallel
>> access. The locking is there just to remind you of this limitation.
> 
> I guess I don't see why OMake should need to lock its database during
> the entire time that a rule is being executed.  If we just need the
> lock to ensure consistency when dependency and job completion
> information is being read or written, then couldn't we just lock the
> database when it's time to read/write that info, and unlock it the
> rest of the time?

The dependency (and other saved information) is being read on startup 
and is then written out in whole at the end of the build (as well as at 
periodic intervals to avoid loosing too much if something goes wrong). 
Currently, neither OMake, nor the .omakedb format has any support for 
updating the information in small increments. This could probably be 
done, but it will be a lot of work and might also result in reduced 
efficiency.

On 17.01.2007 11:17, Yegor Bryukhov wrote:

> Also, OMake can use some third-party DB engines so there is no need to 
> re-implement finer grained locking mechanisms.
> I think, even MySQL now has a version than can be linked into a binary, 
> so users won't have to deal with DB administration.

That assumes that .omakedb is a relational DB. It's not - it's more like 
a partial memory dump (marshalled OCaml value, to be precise). As I 
said, this can be changed, but it will take work and will probably make 
reading/writing the DB slower.

In any case, being able to run more than one instance of OMake on a same 
project is simply not a part of the design goals - the main assumption 
of OMake is that it does a full global project analysis and then has the 
full knowledge of the build process.

If people truly want to be able to spawn extra build tasks while OMake 
is already running, the solution is _not_ to allow a second OMake to 
run; the solution is to have some sort of "OMake Control" protocol for 
telling the existing OMake to do something different. This is already 
implemented in part - if the filesystem monitoring (-p/-P options) is 
enabled and one of the OMakefiles changes, OMake will automatically 
re-read them and adjust the build process accordingly. (*) It should be 
relatively easy to create a way to tell OMake "add target foo to the 
list of targets you are currently building".

Aleksey

---------
(*) On one hand, this can be used to dynamically change the set of 
targets that OMake is trying to build - just run "omake -P" and then 
keep changing the dependencies of the .DEFAULT target. The way it is 
currently implemented, this will not work well in presence of 
long-running jobs - currently, when a change in OMakefiles is detected, 
OMake will first wait for all the currently running jobs to finish and 
only then it will re-read the files.