Research Papers to Markdown (OpenAlex, open access)

Give your agent research papers as clean markdown

Q: What is charged and what is not?

One paper event per paper delivered with its metadata, and one full-text event more per paper delivered with its full text as markdown. IDs that are not found, papers that cannot be read and full texts that cannot be read are not charged. Set a maximum charge per run and it stops before a paper that would pass it.

Q: How are the PDFs read?

As the FrameskimFetch bot, through Apify's datacenter proxy. robots.txt is respected on every request, and sites that ask crawlers to wait between requests get one PDF per run.

Q: How can a site or a rights holder keep its papers out?

Disallow FrameskimFetch in your robots.txt; that also stops the other Frameskim actors that fetch pages. Rights holders can also write to support@frameskim.dev. Neither a robots.txt rule nor a request removes text from datasets that users already hold.

Research Papers to Markdown is an API that returns a paper’s metadata and, when its license allows it, its full text as markdown, for $0.002 per paper plus $0.004 for the full text. Search papers by topic, or look them up by DOI, arXiv ID, PubMed ID or OpenAlex ID, and get one item per paper with its title, authors, year, venue, citations and links and, when its open license allows it, the abstract and full text as markdown. For literature reviews, research agents and RAG over open-access papers.

Run it on Apify View API example

$0.002 per paper delivered, plus $0.004 for each full text. Papers that fail are not charged.

A topic or a list of IDs: each paper, with its full text when the license allows it.

Output, one item per paper

{
  "id": "https://openalex.org/W4389519118",
  "doi": "https://doi.org/10.18653/v1/2023.emnlp-main.495",
  "title": "Active Retrieval Augmented Generation",
  "authors": ["Zhengbao Jiang", "Frank F. Xu", "Luyu Gao"],
  "year": 2023,
  "publicationDate": "2023-01-01",
  "venue": "Conference on Empirical Methods in Natural Language Processing (EMNLP)",
  "type": "conference-paper",
  "citedByCount": 497,
  "isOpenAccess": true,
  "oaStatus": "hybrid",
  "license": "cc-by",
  "landingPageUrl": "https://doi.org/10.18653/v1/2023.emnlp-main.495",
  "pdfUrl": "https://aclanthology.org/2023.emnlp-main.495.pdf",
  "ids": { "openalex": "https://openalex.org/W4389519118", "doi": "https://doi.org/10.18653/v1/2023.emnlp-main.495", "pmid": null, "pmcid": null, "arxiv": null },
  "abstract": "Despite the remarkable ability of large language models...",
  "fullText": "# Active Retrieval Augmented Generation\n\n...",
  "fullTextUrl": "https://aclanthology.org/2023.emnlp-main.495.pdf",
  "fullTextError": null,
  "truncated": false,
  "attribution": ""Active Retrieval Augmented Generation" by Zhengbao Jiang et al. (2023). License: CC BY. Metadata: OpenAlex (CC0)."
}

Values shortened. A missing field is null. license is the license of the open copy the text comes from, as OpenAlex reports it. Without an open license that allows reuse, abstract and fullText are null and fullTextError says why; the paper is still delivered with its metadata.

Use cases

Literature review from Claude or Cursor

Ask a question in plain words: the agent searches the papers, reads in full the open-license ones from sites that allow it and cites each with the attribution its license asks for.

RAG over open-access papers

Clean markdown of open-license papers, with the license OpenAlex reports and the link to the open copy in every item, ready to chunk and index.

Metadata and citations from a list of DOIs

Title, authors, year, venue, citation count and IDs of every paper in one run, with no full text when you only need the record.

Input and output

Input

{
  "query": "retrieval augmented generation",
  "maxResults": 5
}

query searches by topic, title words or author, up to 300 characters. Instead, ids takes DOIs, arXiv IDs, PMIDs, PMCIDs or OpenAlex IDs and the search is not run.

An ID that failed

{
  "input": "10.1234/not-a-paper",
  "error": "No paper was found for this ID. Nothing was charged for it."
}

An ID that could not be delivered comes back with the reason and is not charged.

Pricing

$0.002 per paper,
$0.004 more for each full text

Pay per event on Apify, no subscription: one paper event per paper delivered with its metadata and one full-text event more per paper delivered with its full text as markdown. IDs that are not found, papers that cannot be read and full texts that cannot be read are not charged, and a run never charges more than the maximum you set.

Limits

A search takes a query of up to 300 characters and a maxResults from 1 to 100 (10 by default).
ids takes 1 to 100 IDs per run; a repeated ID is read once.
PDFs up to 5 MB, 15 s per download attempt and 30 s to convert each; 3 papers at a time. A download that runs out of time, cannot connect or gets a server error is tried once more, except on sites that ask for a pause between downloads; if that fails too, the paper comes back without its text and the text is not charged.
Respects robots.txt for FrameskimFetch; scanned PDFs (images only) are not read.

Use it by API or from Claude

API

curl -X POST "https://api.apify.com/v2/acts/emttn~research-papers-markdown/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "retrieval augmented generation", "maxResults": 5}'

MCP

{ "mcpServers": { "apify": {
  "url": "https://mcp.apify.com/?actors=emttn/research-papers-markdown"
} } }

Then ask: "Find open-access papers on retrieval augmented generation and summarize the three most cited."

FAQ

What is charged and what is not?

One paper event per paper delivered with its metadata, and one full-text event more per paper delivered with its full text as markdown. IDs that are not found, papers that cannot be read and full texts that cannot be read are not charged. Set a maximum charge per run and it stops before a paper that would pass it.

Which papers come with full text?

Only papers with an open license that allows reuse: CC BY, CC BY-SA, CC BY-ND, CC0 or public domain. The text is read from the open copy's PDF, and only on sites whose terms allow automated downloads, such as ACL Anthology, PLOS, eLife and Zenodo. For any other site, arXiv and PubMed Central included, you get the link in fullTextUrl and the reason in fullTextError, and the paper is still delivered with its metadata. Scanned PDFs (images only) are not read.

Where does the metadata come from?

From OpenAlex (CC0), which covers Crossref, PubMed and arXiv records. The license of each item is the one OpenAlex reports for the open copy and may be wrong, so check it on the paper's own page before you reuse the text. Each item carries the attribution that license asks for.

How are the PDFs read?

As the FrameskimFetch bot, through Apify's datacenter proxy. robots.txt is respected on every request, and sites that ask crawlers to wait between requests get one PDF per run.

How can a site or a rights holder keep its papers out?

Disallow FrameskimFetch in your robots.txt; that also stops the other Frameskim actors that fetch pages. Rights holders can also write to [email protected]. Neither a robots.txt rule nor a request removes text from datasets that users already hold.