URL to Markdown Conversion API

Overview

The URL to Markdown API allows you to convert any webpage into clean, formatted markdown content. It also extracts metadata like titles, descriptions, and OpenGraph tags.

Convert URL to Markdown

POST

/url-to-markdown

Parameters

Request Body

Parameter	Type	Required	Description
url	string	Yes	The URL of the webpage to convert to markdown

curl -X POST https://api.scrapeloop.com/v1/url-to-markdown \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article"}'

Response

{
  "markdown": "# Example Article\n\nThis is the content of the article converted to markdown...",
  "metadata": {
    "title": "Example Article",
    "favicon": "https://example.com/favicon.ico",
    "language": "en",
    "ogTitle": "Example Article - Example Site",
    "ogDescription": "A great article about examples",
    "ogUrl": "https://example.com/article",
    "ogImage": "https://example.com/article-image.jpg",
    "ogSiteName": "Example Site",
    "publishedTime": "2024-01-24T12:00:00Z",
    "modifiedTime": "2024-01-24T14:30:00Z"
  }
}

Response Fields

Markdown Content

The markdown field contains the converted content with:

• Preserved heading structure
• Formatted lists and tables
• Code blocks with syntax highlighting
• Images and links
• Block quotes and emphasis

Metadata Fields

Field	Type	Description	Example
title	string	Page title	"Example Article"
favicon	string	URL of the page favicon	"https://example.com/favicon.ico"
language	string	Page language	"en"
ogTitle	string	OpenGraph title	"Example Article - Example Site"
ogDescription	string	OpenGraph description	"A great article about examples"
ogUrl	string	OpenGraph URL	"https://example.com/article"
ogImage	string	OpenGraph image URL	"https://example.com/article-image.jpg"
ogSiteName	string	OpenGraph site name	"Example Site"
publishedTime	string	Article published time	"2024-01-24T12:00:00Z"
modifiedTime	string	Article last modified time	"2024-01-24T14:30:00Z"

Response Codes

Code	Description	Possible Cause
200	Success	URL was successfully converted
400	Bad Request	Invalid or missing URL
429	Rate Limit	Too many requests
500	Server Error	Conversion failed

Best Practices

1. URL Validation

Ensure URLs are:

• Properly encoded
• Include protocol (http/https)
• Publicly accessible

function validateUrl(url) {
  try {
    new URL(url);
    return true;
  } catch {
    return false;
  }
}

2. Error Handling

Handle common error scenarios:

async function convertToMarkdown(url) {
  try {
    if (!validateUrl(url)) {
      throw new Error('Invalid URL format');
    }

    const response = await fetch('/url-to-markdown', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'x-api-key': 'YOUR_API_KEY'
      },
      body: JSON.stringify({ url })
    });

    if (!response.ok) {
      throw new Error(`Conversion failed: ${response.statusText}`);
    }

    return await response.json();
  } catch (error) {
    console.error('Conversion error:', error);
    throw error;
  }
}

3. Rate Limiting

Implement rate limit handling:

• Use exponential backoff
• Cache results when possible
• Batch conversions when needed

Examples

Basic Usage

const result = await convertToMarkdown('https://example.com/article');
console.log(result.markdown);

With Metadata

const { markdown, metadata } = await convertToMarkdown('https://example.com/article');

// Use metadata for preview cards
const preview = `
  <div class="preview-card">
    <img src="${metadata.ogImage}" alt="${metadata.title}">
    <h2>${metadata.title}</h2>
    <p>${metadata.ogDescription}</p>
  </div>
`;