SQL incremental sync update using incorrect id columns

XMLWordPrintable

    • Type: Bug
    • Resolution: Invalid
    • Priority: Minor
    • None
    • Affects Version/s: None
    • Component/s: provisioning
    • None

      A full sync of SQL from Grouper groups works fine. But the incremental is trying to use the subject id as the sql table's numeric row id, and failing to convert the string to numeric.

      type: consumer, finalLog: false, state: init, consumerName: provisioner_incremental_McdbGroups, currentSequenceNumber: null, filterInvalidEventTypesSize: 101, publisherClass: edu.internet2.middleware.grouper.app.provisioning.ProvisioningConsumer, runId: v7x13p6i, exception: java.lang.RuntimeException: provisionerClass: SqlProvisioner, configId: McdbGroups, provisioningType: incrementalProvisionChangeLog, state: retrieveIncrementalTargetMemberships, changeLogRawCount: 862, changeLogItemsApplicableByType: 828, recalcEventsDuringFullSync: 0, checkErrorsMinutes: 3, filterNonRecalcActionsCapturedByRecalc: 7, syncGroupsToQuery_Pass1: 246, syncGroupCount_Pass1: 6, retrieveSyncGroupsMillis_Pass1: 4, filterByNotProvisionable: 745, syncMembersToQuery_Pass1: 42, syncMemberCount_Pass1: 42, retrieveSyncMembersMillis_Pass1: 4, convertToFullSyncScore: 118, recalcEventsDuringGroupSync: 0, recalcEventsDuringEntitySync: 0, retrieveDataStartMillisSince1970: 1718981281964, retrieveGrouperGroupsMillis_Pass1: 2, grouperGroupCount_Pass1: 6, retrieveGrouperEntitiesMillis_Pass1: 17, grouperEntityCount_Pass1: 42, retrieveGrouperGroupsEntitiesMillis: 24, convertEntitiesToRecalc: 42, syncMembershipsToQueryFromMember: 42, syncMembershipsFromMember: 148, retrieveSyncMembershipsMillis: 307, syncMembershipCount: 148, retrieveGrouperGroupsMillis_Pass2: 3, grouperGroupCount_Pass2: 6, retrieveGrouperEntitiesMillis_Pass2: 0, grouperEntityCount_Pass2: 0, syncGroupsToQuery_Pass2: 6, syncGroupCount_Pass2: 6, retrieveSyncGroupsMillis_Pass2: 0, syncMembersToQuery_Pass2: 0, syncMemberCount_Pass2: 0, retrieveSyncMembersMillis_Pass2: 0, retrieveGrouperMshipsMillis: 7152, grouperMshipCount: 145, retrieveGrouperGroups2Millis: 0, grouperGroup2Count: 0, retrieveGrouperEntities2Millis: 0, grouperEntity2Count: 0, provisioningMshipsToDelete: 3, retrieveGrouperGroupsMillis_Pass3: 0, grouperGroupCount_Pass3: 0, retrieveGrouperEntitiesMillis_Pass3: 0, grouperEntityCount_Pass3: 0, syncGroupsToQuery_Pass3: 0, syncGroupCount_Pass3: 0, retrieveSyncGroupsMillis_Pass3: 0, syncMembersToQuery_Pass3: 0, syncMemberCount_Pass3: 0, retrieveSyncMembersMillis_Pass3: 0, retrieveGrouperMembershipsMillis: 7153, filterUnneededMemberships: 2, convertMembershipsToRecalc: 146, manipulateAttributesGroupsCount: 12, manipulateAttributesEntitiesCount: 4, duplicateTargetEntities: 2, targetEntitiesRetrieved: 2, originalTargetEntitiesRetrieved: 2, originalTargetTotalCount: 2, retrieveIncrementalTargetGroupsAndEntitiesMillis: 4, calculateGroupActionReset: 12, targetGroupsForLinkNull: 12, provisioningEntityWrappersWithMatch: 4, provisioningEntityWrappersWithNoMatch: 80, linkGcSyncEntitiesUpdated: 2, targetEntitiesForLinkNull: 40,
       
      exception: java.lang.RuntimeException: sql: select groupid, individualid from group_memberships where   (  ( individualid = ?  )  or  ( individualid = ?  )  ... or  ( individualid = ?  )  ) , args: ArrayList size: 42: [0]: 60257915W
      [1]: 11514879Y
      [2]: 11978449W
      [3]: 40867816E
      ,..
      [41]: 91492240B
       
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.callbackResultSet(GcDbAccess.java:2403)
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.selectList(GcDbAccess.java:1765)
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.selectList(GcDbAccess.java:1639)
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.selectList(GcDbAccess.java:1614)
      	at edu.internet2.middleware.grouper.app.sqlProvisioning.SqlProvisionerCommands.retrieveObjectsColumnFilter(SqlProvisionerCommands.java:221)
      	at edu.internet2.middleware.grouper.app.sqlProvisioning.SqlProvisioningDao.retrieveMemberships(SqlProvisioningDao.java:197)
      	at edu.internet2.middleware.grouper.app.sqlProvisioning.SqlProvisioningDao.retrieveMembershipsByEntities(SqlProvisioningDao.java:1573)
      	at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter.retrieveMembershipsByEntitiesHelper(GrouperProvisionerTargetDaoAdapter.java:1689)
      	at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter$9.callLogic(GrouperProvisionerTargetDaoAdapter.java:1784)
      	at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter$9.callLogic(GrouperProvisionerTargetDaoAdapter.java:1778)
      	at edu.internet2.middleware.grouper.util.GrouperUtil.executorServiceSubmit(GrouperUtil.java:14436)
      	at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter.retrieveMembershipsByEntities(GrouperProvisionerTargetDaoAdapter.java:1835)
      	at edu.internet2.middleware.grouper.app.provisioning.targetDao.GrouperProvisionerTargetDaoAdapter.retrieveIncrementalData(GrouperProvisionerTargetDaoAdapter.java:986)
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioningLogicIncremental.retrieveIncrementalTargetMemberships(GrouperProvisioningLogicIncremental.java:2310)
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioningLogic.provisionIncremental(GrouperProvisioningLogic.java:1168)
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioningType$3.provision(GrouperProvisioningType.java:100)
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioningLogic.provision(GrouperProvisioningLogic.java:77)
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioner.provision(GrouperProvisioner.java:855)
      	at edu.internet2.middleware.grouper.app.provisioning.ProvisioningConsumer.dispatchEventList(ProvisioningConsumer.java:91)
      	at edu.internet2.middleware.grouper.changeLog.esb.consumer.EsbConsumer.processChangeLogEntries(EsbConsumer.java:511)
      	at edu.internet2.middleware.grouper.changeLog.ChangeLogHelper.processRecords(ChangeLogHelper.java:406)
      	at edu.internet2.middleware.grouper.changeLog.ChangeLogHelper.processRecordsWrapper(ChangeLogHelper.java:249)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$6.runJob(GrouperLoaderType.java:644)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:554)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob$1.callback(GrouperLoaderJob.java:355)
      	at edu.internet2.middleware.grouper.GrouperSession.callbackGrouperSession(GrouperSession.java:1063)
      	at edu.internet2.middleware.grouper.GrouperSession.internal_callbackRootGrouperSession(GrouperSession.java:1132)
      	at edu.internet2.middleware.grouper.GrouperSession.internal_callbackRootGrouperSession(GrouperSession.java:1099)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:90)
      	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
      	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
      Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: An error occurred during the current command (Done status 0). Error converting data type varchar to numeric.
      	at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:261)
      	at com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer$FetchBufferTokenHandler.onDone(SQLServerResultSet.java:5455)
      	at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:103)
      	at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:42)
      	at com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer.nextRow(SQLServerResultSet.java:5558)
      	at com.microsoft.sqlserver.jdbc.SQLServerResultSet.fetchBufferNext(SQLServerResultSet.java:1821)
      	at com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:1079)
      	at com.mchange.v2.c3p0.impl.NewProxyResultSet.next(NewProxyResultSet.java:685)
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess$2.callback(GcDbAccess.java:1775)
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess$2.callback(GcDbAccess.java:1765)
      	at edu.internet2.middleware.grouperClient.jdbc.GcDbAccess.callbackResultSet(GcDbAccess.java:2400)
      	... 30 more
      , finalLog: true, queryCount: 24, tookMillis: 8855, took: 00:00:08.855
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioner.provisionFinallyBlock(GrouperProvisioner.java:968)
      	at edu.internet2.middleware.grouper.app.provisioning.GrouperProvisioner.provision(GrouperProvisioner.java:887)
      	at edu.internet2.middleware.grouper.app.provisioning.ProvisioningConsumer.dispatchEventList(ProvisioningConsumer.java:91)
      	at edu.internet2.middleware.grouper.changeLog.esb.consumer.EsbConsumer.processChangeLogEntries(EsbConsumer.java:511)
      	at edu.internet2.middleware.grouper.changeLog.ChangeLogHelper.processRecords(ChangeLogHelper.java:406)
      	at edu.internet2.middleware.grouper.changeLog.ChangeLogHelper.processRecordsWrapper(ChangeLogHelper.java:249)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$6.runJob(GrouperLoaderType.java:644)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:554)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob$1.callback(GrouperLoaderJob.java:355)
      	at edu.internet2.middleware.grouper.GrouperSession.callbackGrouperSession(GrouperSession.java:1063)
      	at edu.internet2.middleware.grouper.GrouperSession.internal_callbackRootGrouperSession(GrouperSession.java:1132)
      	at edu.internet2.middleware.grouper.GrouperSession.internal_callbackRootGrouperSession(GrouperSession.java:1099)
      	at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:90)
      	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
      	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
      , tookMillis: 887
      

      rovisioner.McdbGroups.addDisabledFullSyncDaemon = true
      provisioner.McdbGroups.addDisabledIncrementalSyncDaemon = true
      provisioner.McdbGroups.class = edu.internet2.middleware.grouper.app.sqlProvisioning.SqlProvisioner
      provisioner.McdbGroups.customizeMembershipCrud = true
      provisioner.McdbGroups.dbExternalSystemConfigId = mcdb
      provisioner.McdbGroups.deleteMembershipsIfNotExistInGrouper = true
      provisioner.McdbGroups.deleteMembershipsOnlyInTrackedGroups = false
      provisioner.McdbGroups.entityAttributeValueCache0entityAttribute = ucid
      provisioner.McdbGroups.entityAttributeValueCache0has = true
      provisioner.McdbGroups.entityAttributeValueCache0source = grouper
      provisioner.McdbGroups.entityAttributeValueCache0type = entityAttribute
      provisioner.McdbGroups.entityAttributeValueCache1entityAttribute = id
      provisioner.McdbGroups.entityAttributeValueCache1has = true
      provisioner.McdbGroups.entityAttributeValueCache1source = target
      provisioner.McdbGroups.entityAttributeValueCache1type = entityAttribute
      provisioner.McdbGroups.entityAttributeValueCacheHas = true
      provisioner.McdbGroups.entityMatchingAttribute0name = ucid
      provisioner.McdbGroups.entityMatchingAttributeCount = 1
      provisioner.McdbGroups.errorHandlingShow = true
      provisioner.McdbGroups.errorHandlingTargetObjectDoesNotExistIsAnError = false
      provisioner.McdbGroups.groupAllowedToAssign = etc\u003Aprovisioning\u003Aadmin\u003AMcdbGroups_admin
      provisioner.McdbGroups.groupAllowedToView = etc\u003Aprovisioning\u003Aadmin\u003AMcdbGroups_view
      provisioner.McdbGroups.groupAttributeValueCache0groupAttribute = uuid
      provisioner.McdbGroups.groupAttributeValueCache0has = true
      provisioner.McdbGroups.groupAttributeValueCache0source = target
      provisioner.McdbGroups.groupAttributeValueCache0type = groupAttribute
      provisioner.McdbGroups.groupAttributeValueCache1groupAttribute = id
      provisioner.McdbGroups.groupAttributeValueCache1has = true
      provisioner.McdbGroups.groupAttributeValueCache1source = target
      provisioner.McdbGroups.groupAttributeValueCache1type = groupAttribute
      provisioner.McdbGroups.groupAttributeValueCacheHas = true
      provisioner.McdbGroups.groupMatchingAttribute0name = uuid
      provisioner.McdbGroups.groupMatchingAttributeCount = 1
      provisioner.McdbGroups.groupTableIdColumn = uuid
      provisioner.McdbGroups.groupTableName = grouper_groups
      provisioner.McdbGroups.hasTargetEntityLink = true
      provisioner.McdbGroups.hasTargetGroupLink = true
      provisioner.McdbGroups.membershipEntityForeignKeyColumn = individualid
      provisioner.McdbGroups.membershipGroupForeignKeyColumn = groupid
      provisioner.McdbGroups.membershipTableName = group_memberships
      provisioner.McdbGroups.numberOfEntityAttributes = 2
      provisioner.McdbGroups.numberOfGroupAttributes = 3
      provisioner.McdbGroups.numberOfMembershipAttributes = 2
      provisioner.McdbGroups.operateOnGrouperEntities = true
      provisioner.McdbGroups.operateOnGrouperGroups = true
      provisioner.McdbGroups.operateOnGrouperMemberships = true
      provisioner.McdbGroups.provisioningType = membershipObjects
      provisioner.McdbGroups.selectAllEntities = true
      provisioner.McdbGroups.showAdvanced = true
      provisioner.McdbGroups.showAssigningProvisioning = true
      provisioner.McdbGroups.startWith = this is start with read only
      provisioner.McdbGroups.subjectSourcesToProvision = ucmcdb
      provisioner.McdbGroups.targetEntityAttribute.0.name = ucid
      provisioner.McdbGroups.targetEntityAttribute.0.translateExpressionType = grouperProvisioningEntityField
      provisioner.McdbGroups.targetEntityAttribute.0.translateFromGrouperProvisioningEntityField = subjectId
      provisioner.McdbGroups.targetEntityAttribute.1.name = id
      provisioner.McdbGroups.targetEntityAttribute.1.showAdvancedAttribute = true
      provisioner.McdbGroups.targetEntityAttribute.1.showAttributeValueSettings = true
      provisioner.McdbGroups.targetEntityAttribute.1.valueType = int
      provisioner.McdbGroups.targetGroupAttribute.0.name = uuid
      provisioner.McdbGroups.targetGroupAttribute.0.showAdvancedAttribute = true
      provisioner.McdbGroups.targetGroupAttribute.0.showAttributeValueSettings = true
      provisioner.McdbGroups.targetGroupAttribute.0.translateExpressionType = grouperProvisioningGroupField
      provisioner.McdbGroups.targetGroupAttribute.0.translateFromGrouperProvisioningGroupField = id
      provisioner.McdbGroups.targetGroupAttribute.0.valueType = string
      provisioner.McdbGroups.targetGroupAttribute.1.name = name
      provisioner.McdbGroups.targetGroupAttribute.1.translateExpressionType = grouperProvisioningGroupField
      provisioner.McdbGroups.targetGroupAttribute.1.translateFromGrouperProvisioningGroupField = name
      provisioner.McdbGroups.targetGroupAttribute.2.insert = false
      provisioner.McdbGroups.targetGroupAttribute.2.name = id
      provisioner.McdbGroups.targetGroupAttribute.2.showAdvancedAttribute = true
      provisioner.McdbGroups.targetGroupAttribute.2.showAttributeCrud = true
      provisioner.McdbGroups.targetGroupAttribute.2.showAttributeValueSettings = true
      provisioner.McdbGroups.targetGroupAttribute.2.update = false
      provisioner.McdbGroups.targetGroupAttribute.2.valueType = long
      provisioner.McdbGroups.targetMembershipAttribute.0.name = groupid
      provisioner.McdbGroups.targetMembershipAttribute.0.showAdvancedAttribute = true
      provisioner.McdbGroups.targetMembershipAttribute.0.showAttributeValueSettings = true
      provisioner.McdbGroups.targetMembershipAttribute.0.translateExpressionType = grouperProvisioningGroupField
      provisioner.McdbGroups.targetMembershipAttribute.0.translateFromGrouperProvisioningGroupField = groupAttributeValueCache1
      provisioner.McdbGroups.targetMembershipAttribute.0.valueType = int
      provisioner.McdbGroups.targetMembershipAttribute.1.name = individualid
      provisioner.McdbGroups.targetMembershipAttribute.1.showAdvancedAttribute = true
      provisioner.McdbGroups.targetMembershipAttribute.1.showAttributeValueSettings = true
      provisioner.McdbGroups.targetMembershipAttribute.1.translateExpressionType = grouperProvisioningEntityField
      provisioner.McdbGroups.targetMembershipAttribute.1.translateFromGrouperProvisioningEntityField = entityAttributeValueCache1
      provisioner.McdbGroups.targetMembershipAttribute.1.valueType = int
      provisioner.McdbGroups.userPrimaryKey = ucid
      provisioner.McdbGroups.userTableName = individual
      

            Assignee:
            Chris Hyzer (upenn.edu)
            Reporter:
            Chad Redman
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: