FiveTech Software tech support forums

Posted: **Sun Oct 28, 2007 7:10 am**

Hello

I want to write a small program that checks integrity of a dbf file. We have more and more customers having corrupted data in their dbf files.

I have observed the corrupted files and noticed that numeric fields are corrupted with nonsense data but always numeric nonsense, so not easy to track.

The key point is "logical fields" that should contain .t. or .f.
It seems that dbf's integrity is completely broken in these fields where i can find any nonsense chartacter.

I have tried to write a function but i get stuck on reading the logical character

The idea is , read the structure of a dbf file, if the field is loigical test the content of the field and make sure there is .t. or .f.

How can i read the content of the field ?

VALTYPE(...) will always return L for those fields
CVALTOCHAR() willreturn .F. for corrupted fields but not the content

What other function i can use ? Any clue ?

Thanks for your time,

Richard

Posted: **Sun Oct 28, 2007 7:30 am**

Maybe this function could help you.
Regards,
Otto

FUNCTION Translate( cDateiname)
*lMode,oMeter,oOk,oCancel,oDlg,oDbfWnd,lRun)

LOCAL aData
LOCAL bBlock
LOCAL nCounter, nRecno, nStep, nUpd, nFor
LOCAL lAlert
local lMode :=.f.

LOCAL oGaugeDlg
LOCAL oGet1
LOCAL oMeter
LOCAL nActual := 0
LOCAL nTotal := 0
LOCAL cStatusMsg := cDateiname

DbGoBottom()
nTotal := recno()

DEFINE DIALOG oGaugeDlg RESOURCE "DLG_GAUGE" TITLE (" " )
REDEFINE GET oGet1 VAR cStatusMsg ID 2051 OF oGaugeDlg
REDEFINE METER oMeter VAR nActual TOTAL nTotal ID 2052 OF oGaugeDlg

ACTIVATE DIALOG oGaugeDlg ;
CENTERED ;
NOWAIT
oGet1:Refresh()

DbGotop()

IF !lMode
bBlock := {|val,elem| Fieldput(elem,f_isOEM(@val,elem))}
ENDIF

DO WHILE !Eof()
oMeter:Set(nActual+1)
SysRefresh()
aData := Scatter()
Aeval(aData ,bBlock)
dbSkip()

ENDDO

DbUnlock()

oMeter:End()
oGaugeDlg:End()
sysrefresh()

RETURN NIL

FUNCTION scatter(nInicial,nFinal)

local fvals := {} , fnum := 1

nInicial := iif(nInicial == NIL, 1,nInicial )
nFinal := iif(nFinal == NIL, fcount(),nFinal )

Asize(fvals,nFinal-nInicial+1)

aeval(fvals, {|elem| fvals[fnum++] := fieldget(fnum+nInicial-1)})

Return(fvals)

func f_isOEM(val,elem)
local i:=0

msginfo(valtype(elem))

IF valtype(elem)="C"
msginfo(val)
ENDIF
/*
FOR I := 1 TO len(val)

IF ASC(substr(val,i,1)) > 128

VAL:=STRTRAN(VAL,chr(132),"ä")
VAL:=STRTRAN(VAL,chr(148),"ö")
VAL:=STRTRAN(VAL,chr(129),"ü")
VAL:=STRTRAN(VAL,chr(142),"Ä")
VAL:=STRTRAN(VAL,chr(154),"Ü")
VAL:=STRTRAN(VAL,chr(153),"Ö")
VAL:=STRTRAN(VAL,chr(225),"ß")
VAL:=STRTRAN(VAL,chr(179),"ü")
VAL:=STRTRAN(VAL,chr(245),"ä")
VAL:=STRTRAN(VAL,chr(175),"ß")
VAL:=STRTRAN(VAL,chr(247),"ö")
msginfo(VAL+" "+ substr(val,i,1)+" "+ str(ASC(substr(val,i,1))))
ENDIF

*/

return (val)

Posted: **Sun Oct 28, 2007 8:37 am**

Richard

I think it can only be done when each record from the dbf is read with freadstr

Code: Select all

FUNC MAIN(cDbf)
LOCAL nHandle , nPos , nRecSize , nRecords
LOCAL i , el
LOCAL cRecord
LOCAL aStruct
LOCAL aPos[0] , Sum
IF pCount() == 0
  cDbf := "TestDbf"
  DbCreate( cDbf ,{{"String","C",10,0},{"Logical","L",1,0}})
  USE (cDbf)
  APPEND BLANK
  TestDbf->String := "Record  1"
  TestDbf->Logical := .F.
  APPEND BLANK
  TestDbf->String := "Record  2"
  TestDbf->Logical := .T.
  CLOS All
END
USE (cDbf)
nPos     := Header()
nRecSize := RecSize()
nRecords := Reccount()
aStruct := DbStruct()
sum := 0
FOR EACH el IN aStruct
  sum += el[3]
  IF el[2] == "L"
    AADD(aPos,Sum+1)
  END
NEXT
CLOS ALL
nHandle := fopen("TestDbf.dbf")
fSeek(nHandle,nPos)
FOR i := 1 TO nRecords
  cRecord := fReadStr(nHandle,nRecSize)
  ? cRecord
  FOR EACH el In aPos
    ? cRecord[el]
  NEXT
NEXT
WAIT
fclose(nHandle)
RETURN

Posted: **Sun Oct 28, 2007 3:08 pm**

Richard Chidiak wrote:We have more and more customers having corrupted data in their dbf files.

I would search for the real reason. Only three possible causes come up to my mind:

- you are using a bugged xHarbour version
- an hardware/software problem in the customers PC
- a bug in your code

EMG

Posted: **Mon Oct 29, 2007 1:48 pm**

Enrico

- you are using a bugged xHarbour version

The problem is old and i think dbf integrity has always been broken in Harbour / xharbour.

- an hardware/software problem in the customers PC

It is possible but i do not think so, too many hardware problems at one time ?

- a bug in your code

I wish it was so, i could correct it. But how can i dump into dbf something like word dcouments or any other stuff i don't know about ... This is what we usually find in these corrupted files or "cabalistic unreadable characters". This is why i want an inhouse maintenance program that can remove them

Thanks for your concern,

Richard

Posted: **Mon Oct 29, 2007 3:29 pm**

Richard Chidiak wrote:The problem is old and i think dbf integrity has always been broken in Harbour / xharbour.

I never heard of such problems in [x]Harbour RDDs.

EMG

Posted: **Mon Oct 29, 2007 6:04 pm**

Richard,

The corruption may be related to Windows and Novell settings. Here are some old messages about this.

James

---------------------------------
From: "Peter Kohler" <peter.kohler@pixie.co.za>
Subject: Dbf / Cdx corruption
Date: Monday, November 17, 2003 4:40 AM

We have been and are still experiencing a lot of dbf / cdx database
corruption in our apps.
We know this is related to oplocks and client caching.
We now turn off oplcocks and cacheing via the registry settings below

However we are still having problems at clients that use XP workstations
XP seems to cache data and we seem to not have found the correct settings to
turn this off

Can anybody comment on any other registry settings thay they use to prevent
dbf/cdx problems?

Thanks
Peter

***************************************
WINDOWS CLIENT OPLOCK ENTRIES
+ Windows NT/2000/XP clients only
1) HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\LanmanWorkstation\Parameters",
"EnableOpLocks" Sets to 0 to disable oplock requesting by client

2) HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\LanmanWorkstation\Parameters",
"EnableOpLockForceClosed"
Set to 1 to disable oplock file open caching requesting by client

3) Windows 2000/XP clients only (newer setting for Win2K)
HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\MRXSmb\Parameters", "OpLocksDisabled"
Sets to 1 to turn off oplock requesting by client

+ Windows 98 clients only
4) HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\VxD\VREDIR", "DiscardCacheOnOpen"
Sets to 1 to turn off read caching.
Win 98 clients do not know about oplocks and do not use them.

WINDOWS SERVER OPLOCK ENTRIES
+ Win NT/2000/XP/Win2003 server
1) HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\LanmanServer\Parameters",
"EnableOplocks" Set to 0 to disable oplocks

2) HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\LanmanServer\Parameters",
EnableOplockForceClose" Set to 1 to disable oplock open/close
caching

3) HKEY_LOCAL_MACHINE,
"System\CurrentControlSet\Services\LanmanServer\Parameters",
"CachedOpenLimit" Set to 0 to disable oplock open/close caching

--
Peter Kohler
Quick Software SA
peter.kohler@pixie.co.za (home office)
peterk@quicksoftware.co.za (office)
--------------------------------------------------
From: "Rick Lipkin" <lipkinrm@yahoo.com>
Subject: Re: Dbf / Cdx corruption
Date: Monday, November 17, 2003 2:17 PM

Peter

If you are on a Novell 5x or greater server .. put this line in the
Autoexec.ncf:
SET CLIENT FILE CACHING ENABLED = OFF

This will turn off any optlock requests from the clients without having to
dig into each desktop registry.

I am not a M$ server guru .. my guess is that you will have to do thr
regedit thing if your app is on a M$ server.

Rick Lipkin

Posted: **Mon Oct 29, 2007 7:01 pm**

James

Thanks for your concern,

I am aware about oplocks settings, and we do not have "novell" customers. Though we have a big amount of customers running our app, no Novell, just Microsoft .

I still have to find how to read a logical field... It is the solution to cure, Otto's idea is good, scan all characters to check for valid fields, but this might take quite a while, we have some big databases. I will keep searching.

thank you

Richard

FiveTech Software tech support forums

logical character Read

logical character Read

Re: logical character Read

Re: logical character Read

Re: logical character Read