Bulk Subject Lookup: Use grouper_members data first

XMLWordPrintable

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Minor
    • 2.4.0.patch, 2.4.1
    • Affects Version/s: None
    • Component/s: subject API
    • None

      SubjectFinder.findByIds(IdList, SubjectSource) and SubjectResolvers loop over the given IDs and do individual lookups. At Georgia Tech, we had a loader job with about 1M subjects and it took multiple hours to look up the subjects one by one.

      While the Ldap Subject Source Adapter could be optimized to do bulk or other efficient ldap lookups, it seemed that the first method should be to do bulk SQL look ups of matching members in the grouper_members table.

      Doing Member lookups changed the performance from multiple hours to <5 minutes.

            Assignee:
            Shilen Patel
            Reporter:
            Bert Bee-Lindgren
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: