当前位置：首页 -> 基础 -> 数据库编程

Kettle同步设计思路(一)

2014-11-24 08:16:25 · 作者: · 浏览: 14
标签: Kettle 同步 设计 思路

最近在整理之前的同步的kettle代码，现把之前的kettle同步的思路记录在此。

1.同步流程图，下图是同步的整个流程图

\

2.外部的数据表，根据业务逻辑更新到服务器端的业务表，下图是其中的一个示例图：

\

这个的思路是：获取最新需同步的数据、分别查出药品分类ID、药品分类与分类关联ID以及药品ID（不存在，则生成新的UUID）、查询数据库中药品分类ID的记录数是否存在（即是否不为0）、首先插入或更新药品分类表

如果药品分类表中无该条记录，则通过同步标识字段查询出新插入的药品分类ID、插入药品分类关联表、插入药品表；

如果药品分类表中有改条记录，则更新药品分类关联表、更新药品表、更新药品批号表、更新处方表；

下面是这个同步的全部xml：


  
    updateHisDrugData
    
    
    
    Normal
    0
    /updateServerData/commonData
    
    
    





ID_BATCHYID_BATCHCHANNEL_IDYCHANNEL_IDTRANSNAMEYTRANSNAMESTATUSYSTATUSLINES_READYLINES_READLINES_WRITTENYLINES_WRITTENLINES_UPDATEDYLINES_UPDATEDLINES_INPUTYLINES_INPUTLINES_OUTPUTYLINES_OUTPUTLINES_REJECTEDYLINES_REJECTEDERRORSYERRORSSTARTDATEYSTARTDATEENDDATEYENDDATELOGDATEYLOGDATEDEPDATEYDEPDATEREPLAYDATEYREPLAYDATELOG_FIELDYLOG_FIELD







ID_BATCHYID_BATCHSEQ_NRYSEQ_NRLOGDATEYLOGDATETRANSNAMEYTRANSNAMESTEPNAMEYSTEPNAMESTEP_COPYYSTEP_COPYLINES_READYLINES_READLINES_WRITTENYLINES_WRITTENLINES_UPDATEDYLINES_UPDATEDLINES_INPUTYLINES_INPUTLINES_OUTPUTYLINES_OUTPUTLINES_REJECTEDYLINES_REJECTEDERRORSYERRORSINPUT_BUFFER_ROWSYINPUT_BUFFER_ROWSOUTPUT_BUFFER_ROWSYOUTPUT_BUFFER_ROWS






ID_BATCHYID_BATCHCHANNEL_IDYCHANNEL_IDLOG_DATEYLOG_DATELOGGING_OBJECT_TYPEYLOGGING_OBJECT_TYPEOBJECT_NAMEYOBJECT_NAMEOBJECT_COPYYOBJECT_COPYREPOSITORY_DIRECTORYYREPOSITORY_DIRECTORYFILENAMEYFILENAMEOBJECT_IDYOBJECT_IDOBJECT_REVISIONYOBJECT_REVISIONPARENT_CHANNEL_IDYPARENT_CHANNEL_IDROOT_CHANNEL_IDYROOT_CHANNEL_ID






ID_BATCHYID_BATCHCHANNEL_IDYCHANNEL_IDLOG_DATEYLOG_DATETRANSNAMEYTRANSNAMESTEPNAMEYSTEPNAMESTEP_COPYYSTEP_COPYLINES_READYLINES_READLINES_WRITTENYLINES_WRITTENLINES_UPDATEDYLINES_UPDATEDLINES_INPUTYLINES_INPUTLINES_OUTPUTYLINES_OUTPUTLINES_REJECTEDYLINES_REJECTEDERRORSYERRORSLOG_FIELDNLOG_FIELD
    
    
      
      



      0.0
      0.0
    
    10000
    50
    50
    N
    Y
    50000
    Y
    
    N
    1000
    100
    
    
    
    
    
    
    
    
  -
  2012/11/15 14:27:53.963
  -
  2013/02/05 09:53:33.010
  
  
  
  
    backupConn
    localhost
    ORACLE
    Native
    orcl
    1521
    backup
    Encrypted 2be98afc86aa7f2e4cb79ac71dd99baca
    
    
    
    
      FORCE_IDENTIFIERS_TO_LOWERCASEN
      FORCE_IDENTIFIERS_TO_UPPERCASEN
      IS_CLUSTEREDN
      PORT_NUMBER1521
      QUOTE_ALL_FIELDSN
      SUPPORTS_BOOLEAN_DATA_TYPEN
      USE_POOLINGN
    
  
  
    hisConn
    his.com
    ORACLE
    Native
    orcl
    1521
    his
    Encrypted 2be98afc86aa7f2e4cb79ce10be9aa6c9
    
    
    
    
      FORCE_IDENTIFIERS_TO_LOWERCASEN
      FORCE_IDENTIFIERS_TO_UPPERCASEN
      IS_CLUSTEREDN
      PORT_NUMBER1521
      QUOTE_ALL_FIELDSN
      SUPPORTS_BOOLEAN_DATA_TYPEN
      USE_POOLINGN
    
  
  
    serverConn
    server.iwmds.rdh.com
    ORACLE
    Native
    orcl
    1521
    qdias
    Encrypted 2be98afc86aa7f2e4cb79ce61da9baec9
    
    
    
    
      FORCE_IDENTIFIERS_TO_LOWERCASEN
      FORCE_IDENTIFIERS_TO_UPPERCASEN
      IS_CLUSTEREDN
      PORT_NUMBER1521
      QUOTE_ALL_FIELDSN
      SUPPORTS_BOOLEAN_DATA_TYPEN
      USE_POOLINGN
    
  
  
    wardConn
    ward.iwmds.rdh.com
    ORACLE
    Native
    xe
    1521
    ward
    Encrypted 2be98afc86aa7f2e4cb79ce10c993bdde
    
    
    
    
      FORCE_IDENTIFIERS_TO_LOWERCASEN
      FORCE_IDENTIFIERS_TO_UPPERCASEN
      INITIAL_POOL_SIZE50
      IS_CLUSTEREDN
      MAXIMUM_POOL_SIZE500
      PORT_NUMBER1521
      QUOTE_ALL_FIELDSN
      SUPPORTS_BOOLEAN_DATA_TYPEN
      USE_POOLINGY
    
  
  
   值映射数据库查询V_DRUG_CLASS_STRUCT_IDY    增加常量调用DB存储过程V_DRUG_CLASS_IDY    插入 / 更新IWMDS_DRUG_CLASS过滤记录Y    插入 / 更新IWMDS_DRUG_DETAIL更新 IWMDS_DRUG_BATCHY