Locking schemes used with DBF files. Przemyslaw Czerpak (druzus/at/poczta.onet.pl) Exclusive/shared modes (DENY READ, DENY WRITE) in POSIX systems. ================================================================ Exclusive modes (DENY READ, DENY WRITE) do not exist in POSIX systems. They can be emulated by fcntl() or flock() locks. Harbour uses flock() in systems where it does not create conflicts with fcntl() locks used to synchronize file access by different processes in shared mode. Currently it's enabled by default only in Linux (tests with *BSD systems and macOS showed that it interacts with fcntl locks causing deadlocks). On other POSIX systems Harbour uses fcntl() read and write locks at address 0x7fffffff (size 1) to simulate shared and exclusive mode. CLIP and FlagShip use fcntl() locks to simulate shared/exclusive modes. CLIP shared/exclusive flag: 0x7fffffff : 1 (the same as Harbour) FlagShip shared/exclusive flag: 0x7ffffbfc : 1 File access synchronization (record and file locking) in shared mode. ===================================================================== FlagShip uses different record and index locking schemes (i.e. for RLOCK it locks record area) and it is not described here. In documentation CLIP can use Clipper like or FoxPro like locking schemes and it can be controlled by SET LOCKSTYLE [TO] ... command or Set( _SET_LOCKSTYLE, 0 | 1 ) function. This switch only changes the RLOCK/FLOCK offset. For: SET LOCKSTYLE [TO] CLIPPER | CLIPPER50 | CLIPPER53 or: Set( _SET_LOCKSTYLE, 0 ) this offset is set to 1'000'000'000. This is header lock address and records are locked at above address + record number. This is compatible with standard Clipper locking. Commands: SET LOCKSTYLE [TO] CLIPPER52 | FOXPRO | SIX or: Set( _SET_LOCKSTYLE, 1 ) set this offset to 0x10000000 keeping the same algorithm for record locking what is incompatible with any other locking schemes known by me. For sure it's not compatible with [V]FP locking (see details about this locking scheme below). For more information look at CLIP source code. (*) In Harbour index read locks are shared on platforms which support shared (read) locks. Clipper DBFNTX locking: ======================= all locks are exclusive (*) DBF HEADER LOCK: @1000000000 : 1 (exclusive) DBF RECORD LOCK: @1000000000 + recNO : 1 (exclusive) DBF FLOCK SIZE: 1000000000 => maximum records: 3'294'967'295 maximum file size in non POSIX systems: 1'000'000'000 NTX READ LOCK: @1000000000 : 1 (exclusive) NTX WRITE LOCK: @1000000000 : 1 (exclusive) In Harbour it's DB_DBFLOCK_CLIPPER. Clipper new DBFNTX locking (ntxlock2.obj): ========================================== all locks are exclusive (*) DBF HEADER LOCK: @4000000000 : 1 (exclusive) DBF RECORD LOCK: @4000000000 + recNO : 1 (exclusive) DBF FLOCK SIZE: 294967295 => maximum records: 294'967'295 maximum file size in non POSIX systems: 4'000'000'000 NTX READ LOCK: @1000000000 : 1 (exclusive) NTX WRITE LOCK: @1000000000 : 1 (exclusive) In Harbour it's DB_DBFLOCK_CLIPPER2. CL52 DBFCDX, SIX3 SIXCDX, SIX3 SIXNSX, [V]FP CDX ================================================ all locks are exclusive (*) when there is no structural index: DBF HEADER LOCK: @0x40000000 : 1 DBF RECORD LOCK: @0x40000000 + record offset : 1 (some implementations locks whole record) DBF FLOCK SIZE: 0x3ffffffd => maximum file size: 1'073'741'823 when structural index is open: DBF HEADER LOCK: @0x7ffffffe : 1 DBF RECORD LOCK: @0x7ffffffe - recNO : 1 DBF FLOCK SIZE: 0x07ffffff => maximum records: 134'217'727 maximum file size in non POSIX systems: 2'013'265'919 index file locking is the same for production and normal indexes: CDX READ LOCK: @0x7ffffffe : 1 CDX WRITE LOCK: @0x7ffffffe : 1 In Harbour it's DB_DBFLOCK_VFP. CL53 DBFCDX, COMIX (hyper locking) ================================== all locks are exclusive (*), in index shared locks are emulated DBF HEADER LOCK: @1000000000 : 1 DBF RECORD LOCK: @1000000000 + recNO : 1 DBF FLOCK SIZE: 1000000000 => maximum records: 1'000'000'000 maximum file size in non POSIX systems: 1'000'000'000 CDX READ LOCK: @0xffff0000 + random value from 0x0 to 0xffff : 1 CDX WRITE LOCK: @0xfffeffff : 0x10001 if fail then @0xfffeffff : 1 prepare the index modification in memory and before writing to index file lock @0xffff0000 : 0x10000 to eliminate starvation effect caused by many readers on each 16th read lock reading process tries to lock write area @0xfffeffff : 1 instead and then sets normal read lock In Harbour it's DB_DBFLOCK_COMIX. Harbour 32-bit locking ====================== all locks are exclusive (*), in index shared locks are emulated DBF HEADER LOCK: @4000000000 : 1 (exclusive) DBF RECORD LOCK: @4000000000 + recNO : 1 (exclusive) DBF FLOCK SIZE: 294967295 => maximum records: 294'967'295 maximum file size in non POSIX systems: 4'000'000'000 CDX READ LOCK: @0xffff0000 + random value from 0x0 to 0xffff : 1 CDX WRITE LOCK: @0xfffeffff : 0x10001 if fail then @0xfffeffff : 1 prepare the index modification in memory and before writing to index file lock @0xffff0000 : 0x10000 to eliminate starvation effect caused by many readers on each 16th read lock reading process tries to lock write area @0xfffeffff : 1 instead and then sets normal read lock In Harbour it's DB_DBFLOCK_HB32. Harbour 64-bit locking ====================== all locks are exclusive (*), in index shared locks are emulated DBF HEADER LOCK: @0x7fffffff00000001 : 1 DBF RECORD LOCK: @0x7fffffff00000001 + recNO : 1 DBF FLOCK SIZE: 0xfffffffeUL => maximum records: 4'294'967'294 maximum file size: no limits created by locks CDX READ LOCK: @0x7fffffff00000000 + random value from 0x1 to 0xffff : 1 CDX WRITE LOCK: @0x7fffffff00000000 : 0x10001 if fail then @0x7fffffff00000000 : 1 prepare the index modification in memory and before writing to index file lock @0x7fffffff00000001 : 0x10000 to eliminate starvation effect caused by many readers on each 16th read lock reading process tries to lock write area @0x7fffffff00000000 : 1 instead and then sets normal read lock In Harbour it's DB_DBFLOCK_HB64.