7 Matching Annotations
  1. Jul 2023
  2. developers.google.com developers.google.com
    Robots Meta Tags Specifications | Google Search Central  |  Documentation  |  Google for Developers
    1
    1. kael 19 Jul 2023
      robots.txt html <meta name="robots"/> http http:header=x-robots-tag
    Visit annotations in context

    Tags

    Annotators

    URL

    developers.google.com/search/docs/crawling-indexing/robots-meta-tag
  3. Mar 2023
  4. dev.to dev.to
    Tutorial: Add Sitemap.xml and Robots.txt to Remix Site
    1
    1. kael 29 Mar 2023
      remixjs sitemap robots.txt
    Visit annotations in context

    Tags

    Annotators

    URL

    dev.to/chrisbenjamin/tutorial-add-sitemapxml-and-robotstxt-to-remix-site-4n23
  5. Dec 2022
  6. www.bortzmeyer.org www.bortzmeyer.org
    Blog Stéphane Bortzmeyer: RFC 9309: Robots Exclusion Protocol
    1
    1. kael 29 Dec 2022
      robots.txt http bot cito:cites=urn:ietf:rfc:9309
    Visit annotations in context

    Tags

    Annotators

    URL

    bortzmeyer.org/9309.html
  7. datatracker.ietf.org datatracker.ietf.org
    RFC 9309 - Robots Exclusion Protocol
    1
    1. kael 29 Dec 2022
      robots.txt http bot urn:ietf:rfc:9309 wikipedia:en=Robots.txt
    Visit annotations in context

    Tags

    Annotators

    URL

    datatracker.ietf.org/doc/html/rfc9309
  8. Sep 2022
  9. tuixueqiu.com tuixueqiu.com
    三大Robots.txt用途&教学，Robots.txt常见使用误区 – 推雪球
    1
    1. kevinluo 05 Sep 2022
      Robots.txt
    Visit annotations in context

    Tags

    Annotators

    URL

    tuixueqiu.com/332481.html
  10. May 2022
  11. www.wired.com www.wired.com
    How Asimov's Robot Laws Ended Up on Last.fm's Server
    1
    1. chrisaldrich 13 May 2022

      https://www.wired.com/2010/08/robot-laws/

      Last.fm's robots.txt file includes the following Easter egg:

      Disallow: /harming/humans<br /> Disallow: /ignoring/human/orders<br /> Disallow: /harm/to/self

      read robots.txt robots Isaac Asimov three laws of robotics Last.fm
    Visit annotations in context

    Tags

    Annotators

    URL

    wired.com/2010/08/robot-laws/
  12. Apr 2020
  13. github.com github.com
    samboy/genius-blocker
    1
    1. diegodlh 13 Apr 2020
      never even implemented a way for webmasters to opt out of it,

      There were discussions about opt-out mechanisms in the Web Annotation Working group (see here), but in the end it was decided that further work would be deferred to to a future version of the specifications.

      web annotation opt-out web graffiti robots.txt
    Visit annotations in context

    Tags

    Annotators

    URL

    github.com/samboy/genius-blocker