As the web evolves toward API-driven architectures and serverless functions, raw file indexing is becoming less common for public-facing websites. However, it persists in three key areas:
User-agent: * Disallow: /private/