{"id":1957,"date":"2021-09-08T19:57:35","date_gmt":"2021-09-08T10:57:35","guid":{"rendered":"https:\/\/sirius10.net\/blog\/wordpress\/?p=1957"},"modified":"2021-10-03T11:05:06","modified_gmt":"2021-10-03T02:05:06","slug":"post-1957","status":"publish","type":"post","link":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/2021\/09\/08\/1957\/","title":{"rendered":"\u507d\u306e gogglebot \u3084 bingbot \u306b\u5bfe\u51e6\u3059\u308b"},"content":{"rendered":"\n<p>\u3000googlebot \u3084 binbot \u3092\u8a50\u79f0\u3057\u3066\u30a2\u30af\u30bb\u30b9\u3059\u308b\u3082\u306e\u304c\u3042\u308a\u307e\u3059\u3002\u4eca\u307e\u3067\u653e\u7f6e\u3057\u3066\u3044\u307e\u3057\u305f\u304c\u3001Internal Server Error \u3092\u51fa\u3057\u3066\u304d\u305f\u306e\u3067\u5bfe\u51e6\u3057\u3066\u3044\u3053\u3046\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u30c7\u30fc\u30bf\u306e\u53ce\u96c6<\/h2>\n\n\n\n<p>\u3000googlebot \u3084 bingbot \u3092 UA\uff08\u30e6\u30fc\u30b6\u30a8\u30fc\u30b8\u30a7\u30f3\u30c8\uff09\u304b\u3089\u62fe\u3044\u307e\u3059\u3002UA \u306b Googlebot \u304c\u3042\u308b\u30db\u30b9\u30c8\u3092\u8abf\u3079\u308b\u3053\u3068\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted console\">$ grep \" Googlebot\" \u30ed\u30b0\u30d5\u30a1\u30a4\u30eb | sed -e 's\/\\(^.*\\) - - \\[.*\/\\1\/'\n66.249.68.30\ncrawl-66-249-68-1.googlebot.com\n66.249.68.28\n66.249.68.1\ncrawl-66-249-68-1.googlebot.com\n94.177.118.93\n94.177.118.93\n\u30fb\n\u30fb<\/pre>\n\n\n\n<p>\u3000\u3053\u306e\u4e2d\u3067\u3001googlebot.com \u30c9\u30e1\u30a4\u30f3\u3092\u53d6\u308a\u9664\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted console\">$ grep \" Googlebot\" \u30ed\u30b0\u30d5\u30a1\u30a4\u30eb | sed -e 's\/\\(^.*\\) - - \\[.*\/\\1\/' | grep -v \"googlebot.com$\"\n66.249.68.30\n66.249.68.28\n66.249.68.1\n94.177.118.93\n94.177.118.93\n\u30fb\n\u30fb<\/pre>\n\n\n\n<p>\u3000\u91cd\u8907\u884c\u3092\u307e\u3068\u3081\u3001\u6574\u5217\u3055\u305b\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted console\">$ grep \" Googlebot\" \u30ed\u30b0\u30d5\u30a1\u30a4\u30eb | sed -e 's\/\\(^.*\\) - - \\[.*\/\\1\/' | grep -v \"googlebot.com$\" | sort -u\n66.249.68.1\n66.249.68.10\n66.249.68.28\n66.249.68.30\n66.249.68.7\n94.177.118.93<\/pre>\n\n\n\n<p>\u3000\u305a\u3044\u3076\u3093\u6e1b\u308a\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">IP \u30a2\u30c9\u30ec\u30b9\u3092\u691c\u8a3c\u3059\u308b<\/h2>\n\n\n\n<p>\u3000IP \u30a2\u30c9\u30ec\u30b9\u3092\u305d\u308c\u305e\u308c\u9006\u5f15\u304d\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted console\">$ dig -x 66.249.68.1\n\n; &lt;&lt;&gt;&gt; DiG 9.11.5-P4-5.1+deb10u5-Raspbian &lt;&lt;&gt;&gt; -x 66.249.68.1\n;; global options: +cmd\n;; Got answer:\n;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 21293\n;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 9\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n; COOKIE: 0d3bc36061f4af79a2f1a7186136a52dcc8e3a83de04036f (good)\n;; QUESTION SECTION:\n;1.68.249.66.in-addr.arpa.      IN      PTR\n\n;; ANSWER SECTION:\n1.68.249.66.in-addr.arpa. 59110 IN      PTR     crawl-66-249-68-1.googlebot.com.\n\n;; AUTHORITY SECTION:\n68.249.66.in-addr.arpa. 57159   IN      NS      ns1.google.com.\n68.249.66.in-addr.arpa. 57159   IN      NS      ns3.google.com.\n68.249.66.in-addr.arpa. 57159   IN      NS      ns2.google.com.\n68.249.66.in-addr.arpa. 57159   IN      NS      ns4.google.com.\n\n;; ADDITIONAL SECTION:\nns1.google.com.         35465   IN      A       216.239.32.10\nns2.google.com.         35465   IN      A       216.239.34.10\nns3.google.com.         35465   IN      A       216.239.36.10\nns4.google.com.         35465   IN      A       216.239.38.10\nns1.google.com.         35465   IN      AAAA    2001:4860:4802:32::a\nns2.google.com.         35465   IN      AAAA    2001:4860:4802:34::a\nns3.google.com.         35465   IN      AAAA    2001:4860:4802:36::a\nns4.google.com.         35465   IN      AAAA    2001:4860:4802:38::a\n\n;; Query time: 0 msec\n;; SERVER: 127.0.0.1#53(127.0.0.1)\n;; WHEN: \u706b  9\u6708 07 08:33:01 JST 2021\n;; MSG SIZE  rcvd: 381\n<\/pre>\n\n\n\n<p>\u3000\u30cd\u30fc\u30e0\u30b5\u30fc\u30d0\u306b\u3088\u3063\u3066\u5185\u5bb9\u306f\u9055\u3046\u3068\u601d\u3044\u307e\u3059\u304c\u3001ANSWER SECTION \u3092\u898b\u308b\u3068\u3001\u3053\u306e\u30db\u30b9\u30c8\u3082 googlebot.com \u306e\u30db\u30b9\u30c8\u306a\u306e\u3067\u554f\u984c\u306a\u3044\u30db\u30b9\u30c8\u3067\u3059\u300266.249.68.10 \u3068 66.249.68.28\u300166.249.68.30\u300166.249.68.7 \u3082 googlebot.com \u306e\u30db\u30b9\u30c8\u3067\u3057\u305f\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted console\">$ dig -x 94.177.118.93\n\n; &lt;&lt;&gt;&gt; DiG 9.11.5-P4-5.1+deb10u5-Raspbian &lt;&lt;&gt;&gt; -x 94.177.118.93\n;; global options: +cmd\n;; Got answer:\n;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NXDOMAIN, id: 43424\n;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n; COOKIE: fe4f617275470e737a2007836136a728bb25ea085c403651 (good)\n;; QUESTION SECTION:\n;93.118.177.94.in-addr.arpa.    IN      PTR\n\n;; AUTHORITY SECTION:\n118.177.94.in-addr.arpa. 2618   IN      SOA     ns8.heficed.com. postmaster.ns8.host1plus.com. 2021072000 28800 7200 2419200 86400\n\n;; Query time: 0 msec\n;; SERVER: 127.0.0.1#53(127.0.0.1)\n;; WHEN: \u706b  9\u6708 07 08:41:28 JST 2021\n;; MSG SIZE  rcvd: 159<\/pre>\n\n\n\n<p>\u3000\u3053\u306e\u30db\u30b9\u30c8\u306f\u9006\u5f15\u304d\u3067\u304d\u306a\u3044\u306e\u3067 Google \u306e bot \u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002host1plus.com \u306f\u30af\u30e9\u30a6\u30c9\u4e8b\u696d\u3092\u3084\u3063\u3066\u3044\u308b\u3088\u3046\u3067\u3059\u3002\u305d\u306e\u4e2d\u306e\u30db\u30b9\u30c8\u304c\u60aa\u3055\u3092\u3057\u3066\u3044\u308b\u306e\u3067\u3057\u3087\u3046\u3002UA \u3092\u507d\u88c5\u3059\u308b\u306e\u306f\u60aa\u8cea\u3060\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u3000\u507d\u88c5\u3059\u308b IP \u3092\u62bd\u51fa\u3059\u308b SHELL \u30b9\u30af\u30ea\u30d7\u30c8\u3092\u4f5c\u308a\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<p>fake-bot<\/p>\n\n\n\n<pre class=\"wp-block-preformatted file\">#!\/bin\/bash\n\ntrap 'rm -f ${tmpfile}; exit 1' 1 2 3 15\n\ntmpfile=\/tmp\/fake_bot$$\nif [ -f \"${tmpfile}\" ]; then\n    rm -f \"${tmpfile}\"\nfi\n\napache_log=\"\u30ed\u30b0\u30d5\u30a1\u30a4\u30eb\u540d\"\nsafe_domain=\"googlebot.com\"\nua=\"Googlebot\"\n\nif [ $# -eq 2 ]; then\n    ua=$1\n    safe_domain=$2\nelif [ $# -ne 0 ]; then\n    echo \"Usage: $0 [&lt;ua&gt; &lt;safe domain&gt;]\"\n    exit\nfi\n\nfor log in ${apache_log}\ndo\n    grep \"${ua}\" \/var\/log\/apache2\/${log} | sed -e 's\/\\(^.*\\) - - \\[.*\/\\1\/' | gre\np -v \"${safe_domain}$\" &gt;&gt; ${tmpfile}\ndone\n\nfor ip in $(sort -u ${tmpfile})\ndo\n    ptr=$(dig -x ${ip} | grep \"^[^;].*PTR\")\n    domain=${ptr##*PTR  }\n    if [ \"${domain}\" != \"\" ]; then\n        if\n            ! echo ${domain} | grep \"${safe_domain}.$\" &gt;\/dev\/null\n        then\n            echo \"Fake $ua ${ip} ${domain}\"\n        fi\n    else\n        echo \"Fake $ua ${ip}\"\n    fi\ndone\n\nif [ -f \"${tmpfile}\" ]; then\n    rm -f \"${tmpfile}\"\nfi<\/pre>\n\n\n\n<p>\u3000apache_log \u306f\u62bd\u51fa\u3057\u305f\u3044\u30d5\u30a1\u30a4\u30eb\u540d\u3092\u66f8\u304d\u307e\u3059\u3001\u8907\u6570\u3042\u308c\u3070\u3001\u7a7a\u767d\u3067\u533a\u5207\u3063\u3066\u6307\u5b9a\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u3000\u62bd\u51fa\u3057\u305f IP \u3092\u5f3e\u304f\u306b\u306f\u3001\u30d5\u30a1\u30a4\u30e4\u30fc\u30a6\u30a9\u30fc\u30eb\u3067\u5f3e\u304f\u304b\u3001Apache \u306e .htaccess \u3067\u5f3e\u304f\u304b\u306e\u3069\u3061\u3089\u304b\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u3000\u4eca\u56de\u306f\u3001.htaccess \u3067\u5f3e\u304f\u3053\u3068\u306b\u3057\u307e\u3057\u305f\u3002.htsccess \u3067\u5f3e\u304f\u3068\u3001403 \u304c\u8fd4\u308b\u306e\u3067\u3001\u4f55\u5ea6\u3082\u30a2\u30af\u30bb\u30b9\u304c\u3042\u308b\u3088\u3046\u306a\u3089\u3001fail2ban \u3067\u62d2\u5426\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">.htsccess \u3067\u5bfe\u51e6\u3059\u308b<\/h2>\n\n\n\n<p>\u3000.htsccess \u306b\u6b21\u3092\u8ffd\u52a0\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-preformatted file\">&lt;RequireAll&gt;\n  Require all granted\n  Require not ip 94.177.118.93\n&lt;\/RequireAll&gt;<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\u81ea\u52d5\u5316\u3059\u308b<\/h2>\n\n\n\n<p>\u3000\u81ea\u52d5\u3067\u884c\u3046\u306b\u306f\u3001.htaccess \u3092\u7de8\u96c6\u3057\u306a\u3051\u308c\u3070\u306a\u308a\u307e\u305b\u3093\u3002 SHELL \u30b9\u30af\u30ea\u30d7\u30c8\u3092\u4f5c\u6210\u3057\u307e\u3059\u304c\u3001\u81ea\u5206\u306e\u74b0\u5883\u3067\u3057\u304b\u52d5\u304f\u3053\u3068\u3092\u78ba\u8a8d\u3057\u3066\u3044\u306a\u3044\u306e\u3067\u3001\u305d\u306e\u307e\u307e\u4f7f\u3046\u3053\u3068\u306f\u907f\u3051\u305f\u307b\u3046\u304c\u3044\u3044\u3068\u601d\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u3000\u524d\u63d0\u3068\u3057\u3066\u3001.htaccess \u306b RequireAll \u304c\u3042\u308b\u3082\u306e\u3068\u3057\u3066\u3044\u307e\u3059\u3002fake_bot \u306b\u8ffd\u52a0\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p>(2021.10.03 \u8ffd\u8a18)<br>\u3000\u30b9\u30af\u30ea\u30d7\u30c8\u306e\u5185\u5bb9\u306b\u4e0d\u5177\u5408\u304c\u3042\u308a\u4fee\u6b63\u3057\u307e\u3057\u305f\u3002\u30b9\u30af\u30ea\u30d7\u30c8\u306f\u4ee5\u4e0b\u306e\u8a18\u4e8b\u3092\u53c2\u7167\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-wp-embed is-provider-sirius-\u306e\u30d6\u30ed\u30b0 wp-block-embed-sirius-\u306e\u30d6\u30ed\u30b0\"><div class=\"wp-block-embed__wrapper\">\n<blockquote class=\"wp-embedded-content\" data-secret=\"frzxuAQ1Jx\"><a href=\"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/2021\/10\/03\/2120\/\">fake_bot \u3092\u4fee\u6b63<\/a><\/blockquote><iframe loading=\"lazy\" class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" title=\"&#8220;fake_bot \u3092\u4fee\u6b63&#8221; &#8212; Sirius \u306e\u30d6\u30ed\u30b0\" src=\"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/2021\/10\/03\/2120\/embed\/#?secret=SlfcPeVQoS#?secret=frzxuAQ1Jx\" data-secret=\"frzxuAQ1Jx\" width=\"500\" height=\"282\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe>\n<\/div><figcaption>fake-bot<\/figcaption><\/figure>\n\n\n\n<p>(\u8ffd\u8a18\u3053\u3053\u307e\u3067)<\/p>\n\n\n\n<pre id=\"file\" class=\"wp-block-preformatted file\"><strike>!\/bin\/bash\n\ntrap 'rm -f ${tmpfile}; exit 1' 1 2 3 15\n\ntmpfile=\/tmp\/fake_bot$$\nif [ -f \"${tmpfile}\" ]; then\n    rm -f \"${tmpfile}\"\nfi\n\n<span style=\"background-color: #adff2f\" class=\"background-color\">edit_htaccess () {\n    newip=$1\n    htaccess_list=\"\u30d5\u30eb\u30d1\u30b9\/.htaccess\"\n\n    for htaccess in ${htaccess_list}\n    do\n        target=\"${htaccess}\"\n        if\n            ! grep \"${newip}\" \"${target}\" &gt; \/dev\/null\n        then\n            echo \"Add new deny ${newip} to ${htaccess}.\"\n            sed -e \"s\/^\\(&lt;\\\/RequireAll&gt;\\)$\/  Require not ip ${newip}\\n\\1\/\" -i \"${target}\"\n        fi\n    done\n}<\/span>\n\napache_log=\"\u30ed\u30b0\u30d5\u30a1\u30a4\u30eb\u540d\"\nsafe_domain=\"googlebot.com\"\nua=\"Googlebot\"\n\nif [ $# -eq 2 ]; then\n    ua=$1\n    safe_domain=$2\nelif [ $# -ne 0 ]; then\n    echo \"Usage: $0 [&lt;ua&gt; &lt;safe domain&gt;]\"\n    exit\nfi\n\nfor log in ${apache_log}\ndo\n    grep \"${ua}\" \/var\/log\/apache2\/${log} | sed -e 's\/\\(^.*\\) - - \\[.*\/\\1\/' | grep -v \"${safe_domain}$\" &gt;&gt; ${tmpfile}\ndone\n\nfor ip in $(sort -u ${tmpfile})\ndo\n    ptr=$(dig -x ${ip} | grep \"^[^;].*PTR\")\n    domain=${ptr##*PTR  }\n    if [ \"${domain}\" != \"\" ]; then\n        if\n            ! echo ${domain} | grep \"${safe_domain}.$\" &gt;\/dev\/null\n        then\n            echo \"Fake ${ua} ${ip} ${domain}\"\n<span style=\"background-color: #adff2f\" class=\"background-color\">            edit_htaccess ${ip}<\/span>\n        fi\n    else\n        echo \"Fake $ua ${ip}\"\n<span style=\"background-color: #adff2f\" class=\"background-color\">        edit_htaccess ${ip}<\/span>\n    fi\ndone\n\nif [ -f \"${tmpfile}\" ]; then\n    rm -f \"${tmpfile}\"\nfi<\/strike><\/pre>\n\n\n\n<p>\u3000htaccess_list \u306b\u306f\u3001\u30d5\u30eb\u30d1\u30b9\u3067 .htaccess \u3092\u66f8\u3044\u3066\u304f\u3060\u3055\u3044\u3002\u8907\u6570\u3042\u308c\u3070\u7a7a\u767d\u3067\u533a\u5207\u3063\u3066\u66f8\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u3000\u3053\u308c\u3092 cron \u306b\u767b\u9332\u3059\u308c\u3070\u81ea\u52d5\u3067\u8ffd\u52a0\u3057\u3066\u304f\u308c\u307e\u3059\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u3000googlebot \u3084 binbot \u3092\u8a50\u79f0\u3057\u3066\u30a2\u30af\u30bb\u30b9\u3059\u308b\u3082\u306e\u304c\u3042\u308a\u307e\u3059\u3002\u4eca\u307e\u3067\u653e\u7f6e\u3057\u3066\u3044\u307e\u3057\u305f\u304c\u3001Internal Server Error \u3092\u51fa\u3057\u3066\u304d\u305f\u306e\u3067\u5bfe\u51e6\u3057\u3066\u3044\u3053\u3046\u3068\u601d\u3044\u307e\u3059\u3002 \u30c7\u30fc\u30bf\u306e\u53ce\u96c6 \u3000google [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-1957","post","type-post","status-publish","format-standard","hentry","category-server-config"],"_links":{"self":[{"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/1957","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/comments?post=1957"}],"version-history":[{"count":5,"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/1957\/revisions"}],"predecessor-version":[{"id":2144,"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/posts\/1957\/revisions\/2144"}],"wp:attachment":[{"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/media?parent=1957"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/categories?post=1957"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sirius10.net\/blog\/wordpress\/index.php\/wp-json\/wp\/v2\/tags?post=1957"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}