The 'filelumis' REST API can sometimes respond with null event_count

[amaltaro]

While investigating some datasets from central production, I noticed that the filelumis REST API sometimes provides the wrong information.

For instance, this filelumis query gives me the following response:

[
{"event_count":null,"logical_file_name":"/store/mc/Run3Summer22NanoAODv12/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_v5-v4/40000/9b81675a-8a98-476f-8482-8a061d054adb.root","lumi_section_num":7,"run_num":1}
]

Now if I query for this exact LFN in the files REST API, for instance this files query, I get the following response:

[
{"adler32":"f22b0f76","auto_cross_section":0,"block_id":35323312,"block_name":"/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/Run3Summer22NanoAODv12-130X_mcRun3_2022_realistic_v5-v4/NANOAODSIM#f98c387a-9206-4367-8991-c1e181f4b0d2","branch_hash_id":null,"check_sum":"4025616205","create_by":"WMAgent","creation_date":1707861820,"dataset":"/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/Run3Summer22NanoAODv12-130X_mcRun3_2022_realistic_v5-v4/NANOAODSIM","dataset_id":14895314,"event_count":712,"file_id":12096384677,"file_size":5133026,"file_type":"EDM","file_type_id":1,"is_file_valid":1,"last_modification_date":1707861820,"last_modified_by":"WMAgent","logical_file_name":"/store/mc/Run3Summer22NanoAODv12/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_v5-v4/40000/9b81675a-8a98-476f-8482-8a061d054adb.root","md5":null}
]

and we can see that their event_count is different.

[vkuznet]

@amaltaro I doubt it is dbs2go issue since it seems to me it is empty values in DBS database itself. I looked up this LFN directly in DBS DB, and here is what I got

SQL> select F.FILE_ID, F.LOGICAL_FILE_NAME from cms_dbs3_prod_global_owner.files F where F.logical_file_name = '/store/mc/Run3Summer22NanoAODv12/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_v5-v4/40000/9b81675a-8a98-476f-8482-8a061d054adb.root';
 12096384677 /store/mc/Run3Summer22NanoAODv12/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_v5-v4/40000/9b81675a-8a98-476f-8482-8a061d054adb.root
SQL> SELECT DISTINCT FL.RUN_NUM as RUN_NUM, FL.LUMI_SECTION_NUM as LUMI_SECTION_NUM, FL.EVENT_COUNT as EVENT_COUNT from cms_dbs3_prod_global_owner.files F JOIN cms_dbs3_prod_global_owner.file_lumis FL ON F.FILE_ID = FL.FILE_ID where F.file_id=12096384677;
         1                7

or to better see if event_count is not present I used the following SQL:

SQL> SELECT DISTINCT FL.RUN_NUM as RUN_NUM, FL.LUMI_SECTION_NUM as LUMI_SECTION_NUM, coalesce(FL.EVENT_COUNT, -1) from cms_dbs3_prod_global_owner.files F JOIN cms_dbs3_prod_global_owner.file_lumis FL ON F.FILE_ID = FL.FILE_ID where F.file_id=12096384677;
         1                7                          -1

1 row selected.

So, as you can see the event count does not exist for this LFN and therefore the output of dbs2go is correct for filelumis API. In other words, it is issue with data present in DBS and not with dbs2go API.

[todor-ivanov]

Since @vkuznet has checked the validity of the data persisted in DBS and the consistency of of the result returned with the output of the sql queries in the code, I'll not consider this issue as a bug. I am closing it now. @amaltaro feel free to add more info if you consider there is strong necessity for further investigations and improvements.

[amaltaro]

@todor-ivanov I am afraid I have to reopen this issue then for 2 reasons:

1. because the explanation above is wrong and was performed against different file ids
2. as you can see in the links I shared in my initial description, DBS does know that event_count=712, see https://cmsweb.cern.ch/dbs/prod/global/DBSReader/files?logical_file_name=/store/mc/Run3Summer22NanoAODv12/ZPrime2DarkPhoton_HMass-1000_DPMass-0p3_TuneCP5_13p6TeV-pythia8/NANOAODSIM/130X_mcRun3_2022_realistic_v5-v4/40000/9b81675a-8a98-476f-8482-8a061d054adb.root&detail=true .

With that said, if filelumis API is supposed to return an event_count, it should be correct. Otherwise that information should be removed from the results to avoid misleading clients.