Skip to content

updating the way tikautils fetches file#2292

Open
truhacevkir wants to merge 4 commits into
mainfrom
NP-49405-file-download-scopus-import-update
Open

updating the way tikautils fetches file#2292
truhacevkir wants to merge 4 commits into
mainfrom
NP-49405-file-download-scopus-import-update

Conversation

@truhacevkir

@truhacevkir truhacevkir commented Jul 2, 2025

Copy link
Copy Markdown
Collaborator

No description provided.

@github-actions

github-actions Bot commented Jul 2, 2025

Copy link
Copy Markdown

Test Results

  272 files  ±0    272 suites  ±0   21m 55s ⏱️ - 1m 17s
8 780 tests ±0  8 770 ✅ ±0  10 💤 ±0  0 ❌ ±0 
9 311 runs  ±0  9 301 ✅ ±0  10 💤 ±0  0 ❌ ±0 

Results for commit 0f78ba8. ± Comparison against base commit d67679c.

This pull request removes 183 and adds 24 tests. Note that renamed tests count towards both.
            "composer" : "DtePAk2ZjLw8Lzr",
            "day" : "3syDklZHbnBwWUfV5"
            "day" : "rPHYfsH5VIzz"
            "description" : "Some description"
            "extent" : "GuJRYj8fAOMcQPQSMI"
            "formatted" : "M-2306-7118-7"
            "introduction" : {
            "month" : "agqDekENHtR2Gy",
            "month" : "dmslbxWc3hmaUBQiv",
            "name" : "OVuxPTXnSLd7jcg"
…
no.unit.nva.LiteraryArtsAudioVisualMigrationTest ‑ [1] {
          "type" : "LiteraryArtsAudioVisual",
          "subtype" : "RadioPlay",
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rYgvX1EuiAFLb"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "hJ9lbyvYn0VvE5oYHw",
            "month" : "agqDekENHtR2Gy",
            "day" : "rPHYfsH5VIzz"
          },
          "isbn" : "9780099470434",
          "extent" : 1040853105
        }
no.unit.nva.LiteraryArtsAudioVisualMigrationTest ‑ [2] {
          "type" : "LiteraryArtsAudioVisual",
          "subtype" : "RadioPlay",
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rYgvX1EuiAFLb"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "hJ9lbyvYn0VvE5oYHw",
            "month" : "agqDekENHtR2Gy",
            "day" : "rPHYfsH5VIzz"
          },
          "isbnList" : [ "9780099470434" ],
          "extent" : 1040853105
        }
no.unit.nva.LiteraryArtsAudioVisualMigrationTest ‑ [3] {
          "type" : "LiteraryArtsAudioVisual",
          "subtype" : "RadioPlay",
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rYgvX1EuiAFLb"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "hJ9lbyvYn0VvE5oYHw",
            "month" : "agqDekENHtR2Gy",
            "day" : "rPHYfsH5VIzz"
          },
          "extent" : 1040853105
        }
no.unit.nva.LiteraryArtsAudioVisualMigrationTest ‑ [4] {
          "type" : "LiteraryArtsAudioVisual",
          "subtype" : {            "type" : "Other",
            "description" : "Some description"
          },
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rYgvX1EuiAFLb"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "hJ9lbyvYn0VvE5oYHw",
            "month" : "agqDekENHtR2Gy",
            "day" : "rPHYfsH5VIzz"
          },
          "extent" : 10408…
no.unit.nva.LiteraryArtsAudioVisualMigrationTest ‑ [5] {
          "type" : "LiteraryArtsAudioVisual",
          "subtype" : {
            "type" : "Other"
          },
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rYgvX1EuiAFLb"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "hJ9lbyvYn0VvE5oYHw",
            "month" : "agqDekENHtR2Gy",
            "day" : "rPHYfsH5VIzz"
          },
          "extent" : 1040853105
        }
no.unit.nva.LiteraryArtsMonographMigrationTest ‑ [1] {
          "type" : "LiteraryArtsMonograph",
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rCmadtTtK1WQtRkPEA"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "tqrJ3RsfZm",
            "month" : "dmslbxWc3hmaUBQiv",
            "day" : "3syDklZHbnBwWUfV5"
          },
          "isbn" : "9780099470434",
          "pages" : {
            "type" : "MonographPages",
            "introduction" : {
        …
no.unit.nva.LiteraryArtsMonographMigrationTest ‑ [2] {
          "type" : "LiteraryArtsMonograph",
          "publisher" : {
            "type" : "UnconfirmedPublisher",
            "name" : "rCmadtTtK1WQtRkPEA"
          },
          "publicationDate" : {
            "type" : "PublicationDate",
            "year" : "tqrJ3RsfZm",
            "month" : "dmslbxWc3hmaUBQiv",
            "day" : "3syDklZHbnBwWUfV5"
          },
          "isbnList" : [ "9780099470434" ],
          "pages" : {
            "type" : "MonographPages",
            "introduction" : {
…
no.unit.nva.LiteraryArtsMonographMigrationTest ‑ [3] {
       "isbnList": [
        "9788234709159"
       ],
       "pages": {
        "illustrated": false,
        "type": "MonographPages"
       },
       "publicationDate": {
        "type": "PublicationDate",
        "year": "2020"
       },
       "publisher": {
        "name": "Utgiver",
        "type": "UnconfirmedPublisher"
       },
       "type": "LiteraryArtsMonograph"
      }
no.unit.nva.LiteraryArtsMonographMigrationTest ‑ [4] {
       "pages": {
        "illustrated": false,
        "type": "MonographPages"
       },
       "publicationDate": {
        "type": "PublicationDate",
        "year": "2020"
       },
       "publisher": {
        "name": "Utgiver",
        "type": "UnconfirmedPublisher"
       },
       "type": "LiteraryArtsMonograph"
      }
no.unit.nva.MigrateRoleToRoleObjectTest ‑ [1] {
  "type" : "Contributor",
  "role" : "AcademicCoordinator"
}
…

♻️ This comment has been updated with latest results.

private ScopusFile fetchFileContent(ScopusFile file) throws IOException {
var response = fetchResponseAsInputStream(file.downloadFileUrl());
var inputStream = tikaUtils.fetch(response.request().uri());
var inputStream = TikaInputStream.get(response.body());

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consuming inputstream from response directly instead of fetching uri with TikaInputStream.

@truhacevkir truhacevkir requested a review from ketilaa July 2, 2025 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants