TERADATA 13.10 - DATASTAGE 8.5 Issues with FastLoad functionality

Issue description:
When a job that is loading an empty table through FastLoad Functionality (Teradata Connector) aborts, it causes the mentioned table to be left in an irrecoverable Lock state, which cannot be cleared using an empty FastLoad, and which only workaround is to drop and recreate the table.
System description:
·         IBM Datastage Server 8.5, running on “Red Hat Enterprise Linux Server release 5.6 (Tikanga)”
·         Teradata 13.10.0509 13.10.05.09, running on “Red Hat Enterprise Linux Server release 5.6 (Tikanga)”,
·         IBM Datastage Designer and Director 8.5, running on Win7 Enterprise SP1, on an Intel Core i5-3570 with 4GB RAM

·         Teradata SQL Assistant Version 13.11.0,01(English) with ODBC 13.10.00.05, , running on Win7 Enterprise SP1, on an Intel Core i5 3570 with 4GB RAM
Findings:
Doing some deeper research I found that somehow DataStage seems to be sending all the logging that should be sent to SYSADMIN.FASTLOG table to TEST_L (which is the logging table that I have defined for thisjob) instead, while doing a FastLoad
Workaround:
To deal with this problem, I have found two possible solutions:

Use SYSADMIN.FASTLOG as your logging table on Teradata Connector node. This is the easiest way to avoid your tables being locked beyond recovery, since DataStage will send all logging information to the appropriate table, and you will be able to realease the lock by performing an empty FastLoad.

In case you want to use a different logging table, you could use the below script to unlock your tables once thay have been locked. The script has to be run by a user with access to both, SYSADMIN.FASTLOG ,and your table to be able to perform the insertions.

The script basically takes the information from your logging table, and extra information from the parameters received, and performs an insert on SYSADMING.FASTLOG table. After that, it performs an empty FastLoad on your original table to release it, and finally it drops Error and Log tables used during this process.

#### Shell to Unlock Teradata Connector Fastload locked tables
#### Can be executed from DS Job or console.
#### ----------------------------------------------------------
## Parameters: 
### $1 : Logon data. Format: Server/User,Password
### $2 : Database Name
### $3 : Main Table Name
### $4 : Error E1 Table Name
### $5 : Error E2 Table Name
### $6 : Log Table Name
### $7 : Log file destination path

if [ $# -lt 7 ]; then echo 'Incorrect number of parameters'; exit 1; fi
### PARAMS
LOGONDB=$1
FILELOG=$7$3_DS_FastLoad_Unlocker.log
FILE1=$7DS_FastLoad_Unlocker
FILE2=$7DS_FastLoad_Unlocker2
FILE3=$7DS_FastLoad_Unlocker3
echo "Process Starting"> $FILELOG
### BTEQ START
bteq<<$FILE1>>$FILELOG
.LOGON $LOGONDB;
.SET DEFAULTS
.SET TITLEDASHES OFF
.SET WIDTH 254
SEL DATE,TIME;
/******** ********/
INSERT INTO SYSADMIN.FASTLOG (TableName,DatabaseName,StartDate,StartTime,EndDate,EndTime,UserId,Status,RowCount,TotalAMPs,UpAMPs,DownAMPs,Positioning,HostMachine,WorkStationID,AMPArray)
VALUES(
     '$3',
     '$2',
     (SELECT MAX(RUNDATE) FROM $2.$6),
     (SELECT MAX(RUNTIME) FROM $2.$6),
     (SELECT MAX(RUNDATE) FROM $2.$6),
     (SELECT MAX(RUNTIME) FROM $2.$6),
     (SELECT MAX(USERID) FROM $2.$6),
     'BEGINLOAD',
     0,
     (SELECT MAX(TOTALAMPS) FROM $2.$6),
     (SELECT MAX(UPAMPS) FROM $2.$6),
     (SELECT MAX(DOWNAMPS) FROM $2.$6),
     '030202010acd'xb,--Dummy Value
     NULL,
     NULL,
     (SELECT MAX(AMPARRAY) FROM $2.$6)
); 
.LOGOFF;
.QUIT 0;
$FILE1
if [ $? -ne 0 ]; then 
 echo 'Error processing BTEQ  --->: ' $FILE1 
 exit 1
fi
echo 'FastLoad Start'>> $FILELOG
fastload <<$FILE2>>$FILELOG
LOGON $LOGONDB;
BEGIN LOADING $2.$3 ERRORFILES $2.$4,$2.$5;
END LOADING;
QUIT;
$FILE2
echo $rc >>$FILELOG
if [ $? -ne 0 ] && [ $? -ne 4 ]; then 
 echo 'Error processing FastLoad  --->: '$FILE2
 exit 1
fi  
### BTEQ START
bteq<<$FILE3>>$FILELOG
.LOGON $LOGONDB;
.SET DEFAULTS
.SET TITLEDASHES OFF
.SET WIDTH 254
SEL DATE,TIME;
/******** ********/
SELECT 1 FROM dbc.TablesV WHERE databasename='$2' AND TableName='$4';
.if activitycount=0 then .goto ok
DROP TABLE $2.$4 ;
.label ok
SEL DATE,TIME;
/******** ********/ 
SELECT 1 FROM dbc.TablesV WHERE databasename='$2' AND TableName='$5';
.if activitycount=0 then .goto ok
DROP TABLE $2.$5 ;
.label ok
SEL DATE,TIME;
/******** ********/
SELECT 1 FROM dbc.TablesV WHERE databasename='$2' AND TableName='$6';
.if activitycount=0 then .goto ok
DROP TABLE $2.$6 ;
.label ok
SEL DATE,TIME;
/******** ********/
.QUIT 0
$FILE3
if [ $? -ne 0 ]; then 
 echo 'Error processing BTEQ  --->: ' $FILE3
 exit 1
fi
echo "ALL DONE!">> $FILELOG
exit 0;

I can also provide a walkthrough on how to reproduce the error. Just send me a PM or an email.

Hope this may be useful, as I could not find any information on the subject.
Cheers!

Tags:

Forums: