DBF to CSV - converter 3.1 ( Update )

User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

The next step, I started with the REVERSE-part => CSV to DBF.
Here is the function, to create a DBF from the CSV-headline.
It creates a new file like < __CUSTOMER.dbf > ( double underline from original )
Adding the text-data from the CSV, I'm still working on :


Der nächste Schritt, der REVERSE-teil => CSV nach DBF.
Die Funktion zum Erzeugen der DBF aus der CSV-kopfzeile.
Es wird eine neue Datei erzeugt < __CUSTOMER.dbf > ( doppelt Underline vom Original )
Das Hinzufügen von Text aus der CSV-datei ist in Arbeit :

Code: Select all

FUNCTION MAKE_DBF()
LOCAL oText, aStru := {}, N, X, I, nTimes

// CSV-headline-sample, to create the DBF
// LAST-C-20-0;FIRST-C-20-0;HIREDATE-D-8-0;NOTES1-C-10-0;MEMO-M-10-0;NOTES2-C-30-0
 
oText := TTxtFile():New( cCSV ) // open CSV-file

FOR N = 1 to oText:RECCOUNT() // count textlines
    IF N = 1 // only headline
        nTimes := STRCHARCOUNT( oText:READLINE(), cDELIM ) // count sections of delimiter
        FOR I := 1 TO nTimes // for each delimiter
            cField = STRTOKEN( oText:READLINE(), I, cDELIM )
            // MsgAlert( cField, "Field" )
            X := 1
            FOR X := 1 TO 3   //   -C-20-0
                cName = STRTOKEN( cField, X, "-" )
                //MsgAlert( cName, "Name" )
                X++
                cType = STRTOKEN( cField, X, "-" )
                //MsgAlert( cType, "Type" )
                X++
                nLen  = VAL( STRTOKEN( cField, X, "-" ) )
                //MsgAlert( nLen, "Len" )
                X++
                nDec  = VAL( STRTOKEN( cField, X, "-" ) )
                //MsgAlert( nDec, "Dec" )
                X++
                AADD( aStru, { cName, cType, nLen, nDec } )
            NEXT
        NEXT
        DBCREATE( CFILEPATH( cDBF ) + "__" + cFileNoExt( cDBF ) + ".dbf", aStru ) // uses original name with __
    ELSE
        // Adding records from CSV under construction !!!
    ENDIF
    // MsgInfo( oText:ReadLine() )
    oText:Skip()
NEXT
oText:Close()

RETURN NIL
 
best regards
Uwe :lol:
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

The next release makes it possible,
to create a new DBF with selected fields from a existing CSV-file.

Image

Image

best regards
Uwe :lol:
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
FranciscoA
Posts: 1964
Joined: Fri Jul 18, 2008 1:24 am
Location: Chinandega, Nicaragua, C.A.

Re: DBF to CSV - converter 3.1 ( Update )

Post by FranciscoA »

Hi, Uwe.
Excellent work in progress. I would like to use it when you're done.
Regards
Francisco J. Alegría P.
Chinandega, Nicaragua.

Fwxh1204-MySql-TMySql
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

The final new reverse setup-section < CSV to DBF>.
I'm still working on converting the MULTILINE memo.

Image

Best regards
Uwe :lol:
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
Marc Venken
Posts: 727
Joined: Tue Jun 14, 2016 7:51 am

Re: DBF to CSV - converter 3.1 ( Update )

Post by Marc Venken »

Uwe,

Did the conversion CSV to DBF came in a final version ? Where is de download if it have been finished ?
Marc Venken
Using: FWH 20.08 with Harbour
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

Marc,

the image shows a typical export DBF to CSV from excel
but to bulld a DBF we need complete field-informations about type and size

doing my own export I can create a headline like : LAST-C-40 ( fieldname - type - size )
Exel doesn't show these infos.
A solution could be a text-browse and a xBrowse
One with browsing the CSV and one creating the DBF-structure by Your own in relation to the CSV-fields
A problem : for valtype C and N we have _ for the greatest values to define the fieldsizes.
Next we import the CSV-file.

Another solution just adding the missing infos from inside the editor-preview before building the DBF (nothing extra needed) :?:

HIREDATE-D-8-0;SALARY-N-9-2;LAST-C-20-0;FIRST-C-20-0
18.09.92;5900.00;Simpson;Homer
17.10.84;123700.00;Gibbard;Ceci
23.05.89;82900.00;Kaczocha;Reg
10.10.00;120000.00;Jochum;David

Maybe another idea :?:

a original EXCEL export to CSV without any fieldinfos using different fieldtypes.

Image

regards
Uwe :?:
Last edited by ukoenig on Tue Aug 06, 2019 5:58 pm, edited 1 time in total.
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
Marc Venken
Posts: 727
Joined: Tue Jun 14, 2016 7:51 am

Re: DBF to CSV - converter 3.1 ( Update )

Post by Marc Venken »

This is a code from the forum I used for this :

Maybe someone has a use for this..

Code: Select all

function csvtransform()
LOCAL hCsv, cLine, aLine, nI := 0,aFiles[ADIR("*.csv")]

ADIR("*.csv", aFiles)

nFile = msglist(aFiles )
cFile = aFiles[nFile]
cDbf = STRTRAN(lower(cFile), ".csv", ".dbf")



//  Calculate the max field lengt
//  Caution : If there are CDX-files, dbf ceation will faile !!  erase them first of code it...

IF ( hCsv := fOpen( cFile , 16 ) ) > 0
    HB_FReadLine( hCsv, @cLine, chr( 10 ) )
    aHeader = strtoarr(cLine)
    nlenarray = len(aHeader)
    aFieldcount = array(nLenarray)
    afill(aFieldcount,1)
    nTester = 1
    nTeller = 1
    cdeli = MsgSelect( { ";",","} )

    WHILE HB_FReadLine( hCsv, @cLine, chr( 10 ) ) == 0
      oWnd:SetMsg( "Process data "+str(nTeller++))  // Show progress, but slows down on large files
      if cDeli = ","
         FOR I := 1 TO nLenarray

         if I < nLenarray
           cStr = subStr( cLine, 1, at( [,], cLine ) - 1 )
           cStr = STRTRAN(cStr, '"', '')
         else
           cStr = STRTRAN(cLine, '"', '')
         endif

         nlengte = len(alltrim(cStr))

         if aFieldcount[ I ] < nLengte
            afieldcount[i] = nLengte
         endif

         cLine := subStr( cLine, at( [,], cLine ) + 1 )

         //cLine := subStr( cLine, at( ["], cLine ) + 1 )

         NEXT
      else

         FOR I := 1 TO nLenarray

         if I < nLenarray
           cStr = subStr( cLine, 1, at( [;], cLine ) - 1 )
           cStr = STRTRAN(cStr, '"', '')
         else
           cStr = STRTRAN(cLine, '"', '')
         endif

         nlengte = len(cStr)

         if aFieldcount[ I ] < nLengte
            afieldcount[i] = nLengte
         endif

         cLine := subStr( cLine, at( [;], cLine ) + 1 )

         //cLine := subStr( cLine, at( ["], cLine ) + 1 )

         NEXT

      endif

    ENDDO
    builddbf(aHeader,aFieldcount,cDbf)

    fClose( hCsv )
ELSE
    alert( "BAD LUCK" )
    return
ENDIF
fClose( hCsv )
close all
//msginfo("Conversie afgewerkt")
// fill de database



USE &cDbf NEW EXCL ALIAS HUNT

IF ( hCsv := fOpen( cFile, 16 ) ) > 0
    WHILE HB_FReadLine( hCsv, @cLine, chr( 10 ) ) == 0
        //oWnd:SetMsg( "Process data "+str(nTeller--))  // Show progress, but slows down on large files
        nI ++
        IF nI > 1
            hunt->( dbAppend() )
            //FillFields( cLine ) // met ;
            FillFieldscomma( cLine )

        ENDIF
    ENDDO
    fClose( hCsv )
ELSE
    alert( "BAD LUCK 2" )
    return
ENDIF

xBrowse()
close all
*/
RETURN NIL

STATIC FUNCTION FillFieldscomma( cLine )
LOCAL nJ

nMax = hunt->( fCount() )
FOR nJ := 1 TO nMax
    //cLine := subStr( cLine, at( ["], cLine ) + 1 )
    //hunt->( FieldPut( nJ, subStr( cLine, 1, at( ["], cLine ) - 1 ) ) )
    //cLine := subStr( cLine, at( ["], cLine ) + 1 )


    if nJ < nMax
      cStr = subStr( cLine, 1, at( [,], cLine ) - 1 )
      cStr = STRTRAN(cStr, '"', '')
      cStr = alltrim(cStr)
      hunt->( FieldPut( nJ, cStr ))
      cLine := subStr( cLine, at( [,], cLine ) + 1 )
    else
      cStr = STRTRAN(cLine, '"', '')
      cStr = alltrim(cStr)
      hunt->( FieldPut( nJ, cStr ))  // Process last item, regel
    endif

NEXT
RETURN NIL


 
Marc Venken
Using: FWH 20.08 with Harbour
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

Marc,

I think it can be used for the needed fielsize infos
but 3 unknown functions

aHeader = strtoarr(cLine)
cdeli = MsgSelect( { ";",","} )
builddbf(aHeader,aFieldcount,cDbf)

regards
Uwe :D
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
Marc Venken
Posts: 727
Joined: Tue Jun 14, 2016 7:51 am

Re: DBF to CSV - converter 3.1 ( Update )

Post by Marc Venken »

Here the functions, But FW has much better than these old once !!

Code: Select all


function strtoArr(cMaat)
   Local aTemp:={}
   if !empty(cMaat)
     do while at(",",cMaat) > 0
        AADD(aTemp,substr(cMaat,1,at(",",cMaat)-1))
        cMaat = substr(cMaat,at(",",cMaat)+1)
     enddo
     AADD(aTemp,alltrim(cMaat))
   endif
return aTemp

function Builddbf(adata,Alengte,cDbf)
   LOCAL aStru := {}

  n=0
  for i = 1 to len(aData)
     cField = aData[i]
     if len( cField ) > 7
     n++
     cField := substr( cField, 1, 7 - len ( alltrim( str( n, 0 ) ) ) ) + ;
           padl( alltrim( str( n, 0) ), 2, "0" )
     endif
     aData[i] = cField
  next

//  msginfo(atostr(adata))

   for i = 1 to len(adata)
     cField = STRTRAN(aData[i], '"', '')
     aAdd( aStru, { cField , "C", alengte[i]+1 , 0 } )
   next

   dbCreate( cDbf , aStru )

return

function MsgSelect( aItems, cValue, cTitle )

   local oDlg

   DEFINE FONT oFont NAME "Courier New" SIZE 8,15

   DEFAULT cTitle := "Maak uw keuze"

   DEFINE DIALOG oDlg FROM 5,10 TO 24, 95 font oFont TITLE cTitle

   @  1, 2 LISTBOX cValue ITEMS aItems SIZE 305, 110  OF oDlg

   @ 7, 05 BUTTON "&OK"     OF oDlg SIZE 40, 12  ACTION oDlg:End() DEFAULT
   @ 7, 12 BUTTON "&Cancel" OF oDlg SIZE 40, 12  ACTION ( cValue := "", oDlg:End() )

   ACTIVATE DIALOG oDlg CENTERED

return cValue


 
Marc Venken
Using: FWH 20.08 with Harbour
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

Marc,

thank You it is just to check the logic and
only needed for undefined fieldsizes like CSV from Excel.

regards
Uwe :D
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
Marc Venken
Posts: 727
Joined: Tue Jun 14, 2016 7:51 am

Re: DBF to CSV - converter 3.1 ( Update )

Post by Marc Venken »

If you have a Csv-Dbf converter for testing... Let me know
Marc Venken
Using: FWH 20.08 with Harbour
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

Marc,

This will be the reverse-solution CSV to DBF
without fieldtype and size ( from excel )
First we import the fieldnames from the CSV-file
next we can add these missing data with the xbrowse-structure-solution
before importing the data from the CSV-file

Image

Image

regards
Uwe :?:
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
Marc Venken
Posts: 727
Joined: Tue Jun 14, 2016 7:51 am

Re: DBF to CSV - converter 3.1 ( Update )

Post by Marc Venken »

The Browse will be filled with the headers from the csv I suppose ?

Than we can link (Map) the fields to a dbf
Marc Venken
Using: FWH 20.08 with Harbour
User avatar
ukoenig
Posts: 3981
Joined: Wed Dec 19, 2007 6:40 pm
Location: Germany
Contact:

Re: DBF to CSV - converter 3.1 ( Update )

Post by ukoenig »

Marc,

Yes the fieldinfos are complete included in the headline like You can see in image 2 creating CSV-files with the program.
Only files without these fieldinfos ( like from Excel ) we can add them to xBrowse by viewing the CSV-file.
To get the needed max C-field-size I can use Your function.

regards
Uwe :D
Last edited by ukoenig on Tue Aug 06, 2019 6:26 pm, edited 3 times in total.
Since 1995 ( the first release of FW 1.9 )
i work with FW.
If you have any questions about special functions, maybe i can help.
User avatar
Marc Venken
Posts: 727
Joined: Tue Jun 14, 2016 7:51 am

Re: DBF to CSV - converter 3.1 ( Update )

Post by Marc Venken »

I use this for filling my data for a prestashop shopcart.

I have a dbf (left side) with all fields needed in Prestashop

I have dozens of csv files (converted to dbf with the code above) Each manufacturer has it own csv data...

I map the left dbf with the possible fields in the csv file (the right xbrowse)

Image
Marc Venken
Using: FWH 20.08 with Harbour
Post Reply