Troubleshooting
Ever growing list of tips and techniques from the trenches. This was quickly lifted from the IBM internal RabbitHole website, and there are plenty of mistakes to be corrected on this webpage. Broken links to FAQ items among most frequent, and the easiest to fix. Take a look at the TextFormattingRules to see how simple it is to contribute.Troubleshooting TDI
Before digging into this, you might want to check out the official support site. And should you find a bug, please please open a PMR ( = problem management record). Here is a PMR tutorial as some users experience a wall of procedures in their way.Note that the Migration Issues section will not necessarily contain not problems that are fixed in fixpacks or that are already documented in the official documentation.
- Server
- Config Editor
- AMC
- Migration Issues
- Known Issues
Server
Starting/Installing TDI
Loader
ClassNotFoundException
2005.05.31 Refer to the FAQ entry on loading of classes.Config files
Could not load file <filename> The XML document is newer than version 1.1
2005.05.18 (note that the version number might be different) Problem Scenario: (one or more of)- Starting up TDI just does not work.
- Trying to read a IDI 6.0 file with TDI 5.2.
<MetamergeConfig created="Fri Nov 12 12:48:04 CET 2004" createdBy="Neo" modified="Wed May 04<br> 13:39:00 CEST 2005" modifiedBy="Trinity" version="1.1">Or this for version 5.2:
<MetamergeConfig created="Thu Jul 14 15:33:35 CEST 2005" createdBy="Fred" modified="Thu Jul 14<br>16:51:11 CEST 2005" modifiedBy="Ginger" version="1.0">Note that you cannot use the version attribute of the MetamergeConfig node for tracking you own changes. Use filenames instead (e.g. "MyConfig 20051224c.xml") or doc this in the Description tab of ALs and Components.
Error Handling
TDI does not return an error to the calling application
2005.09.12 Problem Scenario: An application (e.g TIM) is calling TDI, and I want it to know about errors that TDI encounters. TDI does not return an error. I do know TDI is aware of the error, since I log a message in the relevant error hook. However, TDI just continues with the AL, I really want it to stop after having logged the error. Reason/Fix:For the lazy fox:
TDI thinks the error has been handled since you have invoked the error hook.
You need to re-throw the error after having logged.
Details:
All errors in TDI are what is called "exceptions"
and each type of error is represented by a Java class. When an error
occurs, the corresponding exception is "thrown", disrupting normal
program execution. The exception is caught by TDI and the appropriate
Error Hook executed. Check out the TDI Flow Diagrams for more details
(found in the docs - Appendix B of the Reference Guide, with a handy
PDF version available there for download).
If you check the Flow Diagrams, you'll see info there explaining that
if an Error Hook is even just enabled, the error is "swallowed". But in some case you
want to pass the error up to the AssemblyLine level, ultimately aborting the
AL with the error.
One solution is to set the Max. number of errors parameter. This a global setting for the AL.
Or if you want to do something more specific, like only escalating specified errors for certain Connectors. In order to do so you must pass the exception up to the AssemblyLine level for handling. In other words, when an Error Hook catches an error, the script code there must literally throw it upwards to the AL.
For details on what went wrong and where, there is a Entry object that is created when an error occurs. The script variable error always references the most recent version, and it contains a number of Attributes of interest. Check here for more info on the error object:
You can't just throw the error object. JavaScript only lets you throw a string (not an Entry). Handily, the "exception" attribute holds the message from a connected system, OS or library) and is throwable.
throw error.getObject("exception");
Then the error is "escalated" up the error flow. If you happen to
re-throw it from the mode-specific On Error Hook of a Connector (like
Iterator Error for Iterator mode) then the error flow goes to the
Default On Error Hook. Otherwise (or if the Default On Error Hook is
disabled) the error is passed back up to the AssemblyLine itself,
and you get the behavior described above (On Failure Hook or stops).
Of course, you can throw any error message you want:
throw "Connection to DB_NAHQ lost"
Note that when you do a system.skipEntry() or system.ignoreEntry() from
script, these actually throw special TDI flow control exceptions that tell
the AssemblyLine about a change in standard processing behavior. But
these never cause Error Hooks to be invoked.
Connectors
Domino Connectors in general
[NotesChange] Could not establish IIOP session; error: 4457: Could not get IOR from Domino Server
Problem Scenario: I am trying to get a Domino Change Detection Connector to work. I am running IDI v6 on Windows Server 2003 connecting to a separate R6 Domino Server. I have installed the R5 version of the Lotus Notes client on the same box as IDI. I have followed the instructions in the reference guide concerning modifying the path and using notes.jar. When I run the AssemblyLine, I get a prompt to enter the password for the local Notes user.id file but then the AL bombs out being unable to establish an IIOP session with the Domino server. I know that I am using the correct userID and password for the IIOP session because it is working in my ibmdi.Notes connectors that are configured elsewhere. Suggestions:- Make sure the http task is started on the Domino server. This is described in the Connector documentation
HTTP Client
The HTTP client does not send messages correctly.
(2005.04.18) Make sure that you have not used an explicit HHTP parser in you HTTP Client Connector: The HHTP Client constructs HTTP messages out of the box. The HTTP parser is really only interesting to use if you e.g. expect to send or receive XML documents within the HTTP request.HTTP Server
The HTTP parser does not parse the incoming messages correctly.
(2005.04.18) Make sure that you have not used an explicit HHTP parser in you HTTP Server Connector: The HHTP Server parses HTTP messages out of the box. The HTTP parser is really only interesting to use if you e.g. expect to send or receive XML documents within the HTTP request.JDBC (DB2 )
"COM.ibm.db2.jdbc.DB2Exception: CLI0616E Error opening socket. SQLSTATE=08S01"
(2005.02.07) A Server Service named DB2 JDBC Applet Server is required to be running on the Window machine where the DB2 Server is running. If The DB2 JDBC Applet Server service is not running you will get this message. Refer to the FAQ that has more information on connecting to a DB2 server."CLI0616E Error opening socket"
(2005.02.07) The remote DB2 server is not configured for DB2 net driver communications. Refer to the FAQ that has more information on connecting to a DB2 server.JDBC (Oracle)
java.sql.SQLException: ORA-01830: date format picture ends before converting entire input string
( 2005.03.02) If you are getting this when inserting/updating date-fields, you are probably passing the Oracle driver dates as a string that does not match what the driver expects. Problem Scenario: (More detailed information about a situation where this can happen, skip to the Suggestion section if not interested) I have an AssemblyLine with a JDBC Connector in addonly mode which writes some records to an Oracle table with a field of type DATE. Normally I would insert something like;INSERT INTO table1
values
(to_date('2005/03/01 10:05:13','YYYY/MM/DD HH:MI:SS'))
as part of my INSERT query.
However with TDI, I can only do something like this in the output map;
ret.value =
'2005/03/01
10:05:13';
but Oracle fails with the following error;
java.sql.SQLException: ORA-01830: date format picture ends before
converting entire input string
Suggestions:
When dates are supplied as strings (which is what you do here) the TDI
JDBC Connector will attempt to parse the data using the pattern
provided in its 'Date Format' configuration parameter, as explained in
the reference guide.
To debug your problem:
- What is your Data Pattern configuration?
- Find out how TDI sees this field (check in the schema tab of the Connector). A fair guess is that your JDBC driver will convert the Oracle Data type into a java.sql.TimeStamp or java.sql.Date type (and note that there are differences between java.util.Date and java.sql.Date, in terms of precision for example). In case, for example, of a java.sql.Timestamp type, try specifying ret.value = java.sql.Timestamp(java.util.Date().getTime()); and see if this helps. Then you will be able to use ret.value = java.sql.Timestamp(system.parseDate(work.getString("yourDate"), "yyyyMMddHHmmssz").getTime());
- If none of the above helps, turn the Connector into detailed log mode and see whether the Connector is able to get the schema from the database. If not, the Connector does not use prepared statements which makes it less efficient and more error-prone - so you'll have to make sure that the Connector's 'schema' configuration parameter is set correctly.
NT4
java.lang.UnsatisfiedLinkError: can't find library NTMetaData? (libNTMetaData.so)
Question: (2005.02.07) I'm trying to use a simple NT4 Connector to watch for NT user updates. However, when I try to run the AssemblyLine, I'm getting a linker error: java.lang.UnsatisfiedLinkError: can't find library NTMetaData? (libNTMetaData.so) How to fix this: The NT Connector is one of the few that must be run in place on the same machine as the server it is talking to -- like the Domino Users Connector.System Store
[Connection Failed]
(2005.02.07) Question: I cannot get the System Store (using Cloudscape in Server mode) to work. No error messages while starting it up, but I cannot connect to it. TDI works fine when CloudScape? is in embedded mode. How to fix this: This seems to be a bug in Cloudscape related to non-English Locale. It is addressed in Derby, but as for TDI 6.0 and 5.* you will need to switch to English locale on the server.[com.ibm.db2.jcc.a.SQLException: Schema 'SYSIBM' does not exist ]
(2005.02.07) Question: I'm trying to use Cloudscape in networked mode and having issues. I've figured out how to start it up and I'm able to query it with sysinfo and test connection, but when I run TDI and try to open the system store I get an error stating: com.ibm.db2.jcc.a.SQLException: Schema 'SYSIBM' does not exist How do I fix this? Explanation: The reason we get this error is because you are trying to boot a db that was created in embedded mode into a n/w mode server without starting the server using the -ld flag. Please note that for a n/w mode CloudScape? server to open an embedded mode database, the SYSIBM schema MUST be loaded when the cloudscape server is started in n/wed mode. This is a CloudScape? requirement. SYSIBM schema is a special schema loaded by the CloudScape? server. The SYSIBM contains stored prepared statements that return result sets to determine metadata information. If this is not loaded and then we try to open a database that was previously created in embedded mode we get the following error: [com.ibm.db2.jcc.a.SQLException: Schema 'SYSIBM' does not exist ] User Response/Corrective action: To solve this problem start the cloudscape n/w server with the "-ld" flag. e.g: ./dbserver start -p 1527 -ld[ERROR XSDB6: Another instance of Cloudscape may have already booted the database D:\tdi60\CloudScape.]
Explanation: (2005.02.07) CloudScape? will try to prevent two instances of CloudScape? from booting the same database (in this case D:\tdi60\CloudScape). This can happen if you are running two ALs which are trying to update the same CloudScape? db running in embedded mode. This error might also crop up if you have an unclosed connection to the database. User Response/Corrective action: 1. If you want two ALs to update the same CloudScape? db, then the correct mode of CloudScape?, should be n/w mode. Please configure the CloudScape? server to run in n/wed mode (link to the info on configuring CloudScape? in n/wed mode in our admin guide). n/wed mode of operation does not have this limitation. 2. You can work around this by closing the database using the "Store/View System store" and then clicking on the close button. Even if the database is not open, just Opening and closing again thru the "Store/View System store" option will help solve this problem.Config Editor
AMC
Migration Issues
5.2 (and earlier) to 6.0
Config Editor
undefined: Cannot convert null to an object. main.logmsg() does not work is 6.0 in scripted parsers
(2005.03.08) Explanation: (This is a bug in 6.0). If you try to do a interactive Connection command using a scripted Connectors or a scripted Parsers attached to a Connector then you will get the above error if you have used the main object in your Connector. Your AssemblyLine will work fine when you run it, it is only when you interactively use the Connect functionality that it will fail. User Response/Corrective action/Workaround: If you want to test your Scripted Connector or Script Parser from the Config Editor, and you want to log, use standard output through java.lang.System.out.println(msg). This will print messages on the console (in the command window) that the Config Editor was started from. This is exactly the same that used to happen when main was defined, and you used main.logmsg() in the context described above.DSMLV2 EH: Parameters get lost (Port number, Binary attributes and Authentication connector)
2005.03.16 Explanation: Certain parameter-names have been changed in the DSMLV2 EH to make them more conformant with the TDI nomenclature. Unfortunately, the values of the old 5.2 parameters did not get copied to the new (6.0) ones. The good news is that if you inspect your XML file, you will see your old and dear values. The bad news is that you will have to lift them manually (using the ConfigEditor?) to the new values. Here is the list of old/new names:| new names | old names |
| port | Port |
| authConnector | AuthConnector |
| binaryAttributes | extraBiAttributes |
Known Issues
This section is not a complete list of known issues, refer to the official list of APARs for something more complete. The goal of this section is to give you early warnings, but the price is that since they have to be maintained by hand by development, the list might contain old fixed items (status might not necessarily be updated). The upside is that items might be visible here before on the official list of APARs.Server
AssemblyLine Flow
Lookup with no match in a loop component
Usually, a Connector in Lookup mode expects one and only one hit, and will send you to the 'error'-hooks On Multiple Entries or On No Match where you can/must handle the situation. The behaviour in a Loop Component is a little confusing:- On Multiple Entry is never called
- On No Match is called if no match returned by the Lookup Connector.
java.lang.Exception: [IF_MgrFound] Entry not found
and possibly a crash if nothing catches the 'exception'.
This is really a bug: For Lookup Connectors in a loop, the On Multiple Entries or No Entry Found should be called only if they exist, and no fuzz made if they don't.
Workaround if you don't want an exception when no entries returned:
- Enable your No Entry Found without any code in it.
Loop bug when exiting loop (resolved in fixpack2)
2005.10.20 2005.06.10 2005.06.09 If there is code inside the loop that exits the loop prematurely, the next time the AL reaches the loop, the AL will continue where it left off (the next value). That is, the loop does not begin at the beginning, which you would expect. This can happen e.g. when a user calls system.skipEntry(), to go to the next Entry in the iteration, or when using the system.exitBranch() method, which is supposed to be a correct way to exit from a Loop. This problem occurs for all kinds of loops; multivalued attributes, iterator Connectors and lookup Connectors. A workaround is to drain the loop before you exit it. This is not pretty, but a workaround is better than none:
while (MyLoop?.willExecute(work)) {
MyLoop?.add(work);
// go through all
the values this loop would have delivered.
}
Advanced link criteria for a Lookup Connector in a Loop deleted when saving config
2005.10.20 The workaround is to put your advanced Link Crit in a Connector in your Library that you use in the Loop. Then you simply inherit the Link Crit as well.How to programatically disable components (Branches, Connectors etc) using TCB (TDI 6.0 and before)
2005.06.28 This is unfortunately a little tricky. Generally speaking, you will need to use the Initial Work Entry (IWE) to pass a control flag. Of course, if your AL has an Iterator, you need to store this value in a script variable and zero out the Work Entry (or the Iterator will not engage on the first cycle). For example, to disable a branch, you might say something like: var branchEnabled = work.getString( "enableBranch" ); task.setWork( null ); Then set your Branch "Match All" and include a scripted condition like this: ret.value = branchEnabled.equals( "yes" ); Note that if you intend to use your IWE, you must you an extra attribute that you possibly must clear out before continuing. Disabling a Connector is much harder, actually it cannot be done without modifying the config object before you start the AssemblyLine. The problem is that if the Connector exists (not disabled), it will be initialized, even if you disable it in the prolog before initialization. Modifying the in-memory config object is doable, but not advised. What you can do is set your Connector to passive, but this will not help you if you try avoiding an initialization completely. This not a bug, but is recognized by development as a limitation.Command Line Issues
-c switch does not work as advertised in TDI 6.0
2005.06.30 The -c switch does work with multiple filenames. Short story: You must give-d as well to make it work. Long Story: The documentation/functionality would have been correct if it read:
-c <file ....>
Configuration file(s).
If you do not specify the -d switch, only one configuration file is
allowed.
If you don't specify the -c option, the items in the Autostart folder
will be loaded and started (unless suppressed by specifying -D).
Wildcards, as in *.xml, are allowed.
Workaround for the problem. Specify the -d switch (Run in daemon mode)
if you want to use more than one configuration file.
Configuration
log4j.properties example is wrong
2005.08.18 The example on how to set up a specific log strategy is wrong. The log4j.properties file has flawed example lines (TDI 6.0, SP1). These lines are#
log4j.logger.AssemblyLine.myAL=INFO, CONSOLE
# log4j.logger.AssemblyLine.myAL=INFO, fileLOG
They should have been
#
log4j.logger.AssemblyLine.AssemblyLines/myAL=INFO, CONSOLE
# log4j.logger.AssemblyLine.AssemblyLines/myAL=INFO, fileLOG
Delta and Changelog
Commit on Iterator Delta: What if something went wrong after the Delta engine marked the entry processed?
2005.06.06 When iterating from a file or database with delta (iterator) functionality turned on, if the AL is unable to complete the update operation (failure in the target, lacking data that requires a skipEntry, or..), the AL has already tagged the Entry (in the delta store) as processed. So (unless it has changed again) the next run of the AL will not include the Entry in question, despite the fact that its processing was not completed earlier. Here is an example (Source and Target are two repositories to be synchronized, pushing changes from Source to Target. Delta is the delta store used internally by TDI).- Source contains v = 5, and so does the Delta and Target. All is well and in synch.
- Source is modified so v = 42
- Al runs, iterates over Source and sees that Delta has v=5 which is different from Source where v=42
- AL updates the Delta to v=42
- Something goes wrong, and Target does not get updated. Target still has v=5
- Next run, Source has v = 42, and so has Delta. So the delta engine does not trigger a change in Target: Target is stuck with unsynchronized value 5.
Memory Leak
Reinitialization of Connectors
2005.05.27 2005.05.25 If your business logic is so that you reinitialize Connectors a lot, make sure to use their terminate() method before you call their initialize() method. The classic here is an AL starting up but not able to connect to your data source, if you do the reconnection wrong, you might bleed memory. Check out the entry on Reconnecting in the FAQ.JVM
2005.05.20 There are multiple known memory issues with the IBM JVM. These are being addressed swiftly by the JVM team. If you experience memory leaks, make sure you are using the latest JVM.Logging
2005.05.20 TDI Version 5.2 and 6.0 both have a memory leak that will show up in servers where AssemblyLines or EventHandlers are running for an extended time and where the log4j is enabled (it is default). This issues will be addressed in APARs both for 5.2 and 6.0. Until then, the only thing that you can do if this is hitting you, is restarting the server now and then. Restarting the AL/EH is not enough.JavaScript/Rhino
2005.04 See the AMC and JavaScript sales flash.Config Editor (CE)
Config Creation
Save As command not always working properly
2005.07.07 TDI 6.0: The Save As command will not save Autostart and Logconfig information in the configfile. Work-around: Use the shell command copy (or cp) instead of save As if you use logconfig or autostart.External Parameters
Locking of parameter file
2005.07.07 The config editor will not close the file external parameter file after opening it with the CE. Under Windows, that will actually lock the file. Workaround: Close CE before editing the external properties file if this is a problem for you.Sales flash
AMC and JavaScript
- April 2004 (AMC and JavaScript (Rhino) notices. Covered in the release notes (see below)
6.0 Documentation
Latest release notes
First of all, note that the release notes contain errata information, as do the readme files of all fixpacks. Beneath is an attempt at getting you even newer information if I have access to it. Please check what version is newest and let me know if my 'newer' info is outdated.- 2005.04.21 Updated release notes with a warning about the cloudscape security issue.
- 2005.04.20 Updated release notes with a special warning about Rhino R2 proneness to memory leaks and a notice on the fact that AMC2 is indeed supported.
JavaDocs Documentation errors
- 2005.04.21 There should not have been JavaDocs for the JLDAP Connector (that is not part of TDI). This could be the case for other arcane/deprecated Connectors as well.
Updated Documentation
Function Component flow-charts
2005.05.06 The 6.0 Documentation did not document the hook flow of the Function Component. Here is a pdf file that documents it. Also, not that the Update Mode has been corrected (removed current object from right sidebar)Documentation Errors
Connector Hooks: Error Flow
2005.09.08 This is valid for TDI 6.0 and 5.2 documentation. If a Connector fails during initialization, the error hook flow charts say that the following hooks will be called: Initialization On Error Mode Specific On Error Default On Error Only the Initialization On Error hook is called. This is correct, the documentation will be updated. The corresponding situation exists for the Connector Close flow.6.0 Connectors
Connectors: Reconnect on connection lost
2005.10.04 There are two things to note here:- The reconnect functionality (hooks and possible auto-reconnect) only kicks in in the AL flow, not during Connector initialization.
- Checkpoint/Restart(CPR) is independent of Reconnect in 6.0 (this might be changed). So even if you have Checkpoint/Restart information saved, the reconnect functionality will not use it. CPR information will only be used if the AL completely crashes and is restarted.
Connector Modes: Delta
2005.04.18 (TDI 6.0 only) When using Connectors in Delta mode and the operation is delete, there is no mapping of attributes. This means that unless you have specified a link criteria that identifies the record to be deleted (such as dn in LDAP), the Connector will not be able to delete. Here are a couple of workarounds for the LDAP Connector: (this assumes that you have used yourdn as attribute name for distinguished name in work)- Set up a link criteria specifying $dn equals $yourdn
- In the Before Delete hook, set up the mapping explicitly by coding:
conn.setAttribute("$dn",work.getString("yourdn"));
Domino User Connector
Instructions
2005.04.12 If the Domino User Connector does not work as advertised under the Windows Platform, as the documentation is wrong. Here is an updated: 1) In 6.0, there is a new place to update extra jar files for TDI to run (such as Notes.jar...). You want to point the system property com.ibm.di.userjars to where you have Notes.jar located. If you look in either the global.properties or solution.properties file (depending on if you are using a solutions directory), you will see a section near the top of the file that looks like the following.##
## Modify the line below to add your own directory containing jar/zip
files. This directory will be searched recursively
## by the TDILoader for class files and resources. Only files with a
"zip"
or "jar" extension are searched.
##
# com.ibm.di.loader.userjars=c:\myjars
You can either edit this value OR... another way would be to
put
a copy
of the Notes.jar file in a subdirectory that is already in the TDI path
(some people call this 'cheating' or not as clean... whatever).
2. Remove/rename the ncso.jar (as mentioned in the docs)
3. Edit the ibmdisrv.bat and ibmditk.bat file the following
way...
The path variable that is set in the batch files ibmdisrv.bat and
ibmditk.bat needs an ever so subtle but important edit so the Notes.jar
file can see the binaries. Remove the double quotes from the
set
path
statement in the batch files. Windows is effectively ignoring
this path
statement with those double quotes and so when you add your new path to
the domino binaries (as mentioned in the docs... something like
c:\lotus\domino) , TDI is not able to find it.
So. A.
add the new
path....B. remove the double quotes.
JDBC Connector
Handling of CLOB/BLOB (Character/Binary large object)
2005.04.22 If your attributes are of CLOB/BLOB type, the Connector does not handle them on output. On input, you can do something likedesc =
conn.getObject("yourCLOBAttribute");
ret.value =
desc.getSubString(1,desc.length());
but it is slow and clumsy. Also it will only work if the JDBC driver actually returns a java.sql.Blob or java.sql.Clob interface objct.
Check out the the official list of APARs for a fix.
6.0 Function Components
Axis Java To Soap FC
Interopability with AXIS C++ compiler when using complex types.
2005.08.31 The Axis JavaToSoap? Function Component (FC) converts Java object to SOAP messages. The FC can handle complex type and since Axis Java supports href/multiRef, it converts Java objects to SOAP in href/multiRef format by default. However Axis C++ doesn’t support href/multiRef. This limits the interoperability with Java version of Axis. Native Axis can disable href/multiRef feature by putting<parameter
name="sendMultiRefs" value="false"/>
under
<globalConfiguration> in the server-config.wsdd file. TDI
does not have this fields as to configure Axis:
This means TDI 6.0 cannot communicate with Axis C++ using complex types.
This issue will be addressed and you are encouraged to open a PMR if
you have customer need for it. In the meantime, Katsuyuki
Hirayama has created a fix
published on the Xpertise library.
You need to contact the author to get access to it, as we need to be
sure you understand that this is not officially supported.
