tomnomnom/meg là dự án mã nguồn mở trên GitHub với 1.7k sao, viết chủ yếu bằng Go. Fetch many paths for many hosts - without killing the hosts
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
meg is a tool for fetching lots of URLs but still being 'nice' to servers.
It can be used to fetch many paths for many hosts; fetching one path
for all hosts before moving on to the next path and repeating.
You get lots of results quickly, but non of the individual hosts get
flooded with traffic.
Install
meg is written in Go and has no run-time dependencies. If you have Go 1.9
or later installed and configured you can install meg with go install:
▶ go install github.com/tomnomnom/meg@latest
Or download a binary and
put it somewhere in your $PATH (e.g. in /usr/bin/).
Install Errors
If you see an error like this it means your version of Go is too old:
# github.com/tomnomnom/rawhttp
/root/go/src/github.com/tomnomnom/rawhttp/request.go:102: u.Hostname undefined (
type *url.URL has no field or method Hostname)
/root/go/src/github.com/tomnomnom/rawhttp/request.go:103: u.Port undefined (type
*url.URL has no field or method Port)
/root/go/src/github.com/tomnomnom/rawhttp/request.go:259: undefined: x509.System
CertPool
You should either update your version of Go, or use a binary release
for your platform.
▶ meg --verbose paths hosts
out/example.com/45ed6f717d44385c5e9c539b0ad8dc71771780e0 http://example.com/robots.txt (404 Not Found)
out/example.com/61ac5fbb9d3dd054006ae82630b045ba730d8618 https://example.com/robots.txt (404 Not Found)
out/example.net/1432c16b671043271eab84111242b1fe2a28eb98 http://example.net/robots.txt (404 Not Found)
out/example.net/61deaa4fa10a6f601adb74519a900f1f0eca38b7 http://example.net/.well-known/security.txt (404 Not Found)
out/example.com/20bc94a296f17ce7a4e2daa2946d0dc12128b3f1 http://example.com/.well-known/security.txt (404 Not Found)
...
And save the output in a directory called ./out:
▶ head -n 20 ./out/example.com/45ed6f717d44385c5e9c539b0ad8dc71771780e0
http://example.com/robots.txt
> GET /robots.txt HTTP/1.1
> Host: example.com
< HTTP/1.1 404 Not Found
< Expires: Sat, 06 Jan 2018 01:05:38 GMT
< Server: ECS (lga/13A2)
< Accept-Ranges: bytes
< Cache-Control: max-age=604800
< Content-Type: text/*
< Content-Length: 1270
< Date: Sat, 30 Dec 2017 01:05:38 GMT
< Last-Modified: Sun, 24 Dec 2017 06:53:36 GMT
< X-Cache: 404-HIT
<!doctype html>
<html>
<head>
Without any arguments, meg will read paths from a file called ./paths,
and hosts from a file called ./hosts. There will also be no output:
▶ meg
▶
But it will save an index file to ./out/index:
▶ head -n 2 ./out/index
out/example.com/538565d7ab544bc3bec5b2f0296783aaec25e756 http://example.com/package.json (404 Not Found)
out/example.com/20bc94a296f17ce7a4e2daa2946d0dc12128b3f1 http://example.com/.well-known/security.txt (404 Not Found)
You can use the index file to find where the response is stored, but it's
often easier to find what you're looking for with grep:
If you want to request just one path, you can specify it directly as an argument:
▶ meg /admin.php
Detailed Usage
meg's help output tries to actually be helpful:
▶ meg --help
Request many paths for many hosts
Usage:
meg [options] [path|pathsFile] [hostsFile] [outputDir]
Options:
-c, --concurrency <val> Set the concurrency level (defaut: 20)
-d, --delay <val> Milliseconds between requests to the same host (default: 5000)
-H, --header <header> Send a custom HTTP header
-r, --rawhttp Use the rawhttp library for requests (experimental)
-s, --savestatus <status> Save only responses with specific status code
-v, --verbose Verbose mode
-X, --method <method> HTTP method (default: GET)
Defaults:
pathsFile: ./paths
hostsFile: ./hosts
outputDir: ./out
Paths file format:
/robots.txt
/package.json
/security.txt
Hosts file format:
http://example.com
https://example.edu
https://example.net
Examples:
meg /robots.txt
meg -s 200 -X HEAD
meg -c 30 /
meg hosts.txt paths.txt output
Concurrency
By default meg will attempt to make 20 concurrent requests. You can change that
with the -c or --concurrency option:
▶ meg --concurrency 5
It's not very friendly to keep the concurrency level higher than the number of
hosts - you may end up sending lots of requests to one host at once.
Delay
By default meg will wait 5000 milliseconds between requests to the same host.
You can override that with the -d or --delay option:
▶ meg --delay 10000
Warning: before reducing the delay, ensure that you have permission to make
large volumes of requests to the hosts you're targeting.
Adding Headers
You can set additional headers on the requests with the -H or --header
option:
You can use the -r or --rawhttp flag to enable use of the rawhttp
library, which does little to no validation on the request:
▶ meg --verbose --rawhttp /%%0a0afoo:bar
out/example.com/eac3a4978bfb95992e270c311582e6da4568d83d https://example.com/%%0a0afoo:bar (HTTP/1.1 404 Not Found)
The rawhttp library and its use is experimental. Amongst other things it doesn't
yet support chunked transfer encoding, so you may notice chunk lengths interspersed
with your output if you use it.
Saving Only Certain Status Codes
If you only want to save results that returned a certain status code, you can
use the -s or --savestatus option:
▶ meg --savestatus 200 /robots.txt
Specifying The Method
You can specify which HTTP method to use with the -X or --method option:
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/tomnomnom/meg là nguồn chính thức.
tomnomnom/meg có phải mã nguồn mở không?
Có — tomnomnom/meg phát hành theo license MIT, nghĩa là mã nguồn mở để đọc, fork và (tùy license) tái sử dụng. Mã: github.com/tomnomnom/meg.
tomnomnom/meg có website riêng không?
TopGit chưa ghi nhận URL trang chủ cho tomnomnom/meg. Phần README ở tab phía trên thường có link demo, hoặc xem mô tả GitHub của repo.
tomnomnom/meg dùng license gì?
tomnomnom/meg phát hành theo license MIT. Nên mở file LICENSE trên GitHub để xác nhận — license metadata đôi khi lệch với thực tế dự án.
tomnomnom/meg là gì?
tomnomnom/meg (tomnomnom/meg) là dự án Go trên GitHub. Theo mô tả gốc: Fetch many paths for many hosts - without killing the hosts
Đọc đầy đủ README ở tab phía trên.
Chưa chắc meg có hợp với bạn?
Để ChatGPT, Claude hoặc Perplexity tìm hiểu giúp — bấm bên dưới và xem AI nói gì về meg.