Find files with the same content in one folder (and all its sub-folders)
Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).
Is there any way I can do this on Mac?
macos file
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
add a comment |
Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).
Is there any way I can do this on Mac?
macos file
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
Jan 1 at 5:06
add a comment |
Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).
Is there any way I can do this on Mac?
macos file
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
Say I have a folder named container and there are many files/sub-folders in it. I would like to find all the files with the same content (they can have different names but the content should be the same).
Is there any way I can do this on Mac?
macos file
macos file
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
edited Jan 1 at 5:07
Nimesh Neema
15.3k44074
15.3k44074
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
asked Jan 1 at 4:38
AGamePlayerAGamePlayer
3751516
3751516
I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
Jan 1 at 5:06
add a comment |
I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
Jan 1 at 5:06
I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
Jan 1 at 5:06
I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
Jan 1 at 5:06
add a comment |
1 Answer
1
active
oldest
votes
If you're comfortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.
You'll need to install it with Homebrew first: brew install rdfind
Then, let's say you want to run a dry test on your ~/Download folder:
rdfind -n true -outputname result.log ~/Downloads/
-n trueis for dry mode: no files will be affected
-outputname result.logwill output the scanning into that file
~/Downloads/is the argument of the root folder you want to scan
(you can pass more than one folder)
Running that command will look something like that:
$ rdfind -n true -outputname result.log ~/Downloads/
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
(DRYRUN MODE) Now have 20132 files in total.
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
(DRYRUN MODE) It seems like you have 15455 files that are not unique
(DRYRUN MODE) Totally, 324 MiB can be reduced.
(DRYRUN MODE) Now making results file result.log
Or, you can use an application with a graphical interface like dupeguru:
answered Jan 1 at 5:39
YoricYoric
4035
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "118"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fapple.stackexchange.com%2fquestions%2f347217%2ffind-files-with-the-same-content-in-one-folder-and-all-its-sub-folders%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
If you're comfortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.
You'll need to install it with Homebrew first: brew install rdfind
Then, let's say you want to run a dry test on your ~/Download folder:
rdfind -n true -outputname result.log ~/Downloads/
-n trueis for dry mode: no files will be affected
-outputname result.logwill output the scanning into that file
~/Downloads/is the argument of the root folder you want to scan
(you can pass more than one folder)
Running that command will look something like that:
$ rdfind -n true -outputname result.log ~/Downloads/
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
(DRYRUN MODE) Now have 20132 files in total.
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
(DRYRUN MODE) It seems like you have 15455 files that are not unique
(DRYRUN MODE) Totally, 324 MiB can be reduced.
(DRYRUN MODE) Now making results file result.log
Or, you can use an application with a graphical interface like dupeguru:
answered Jan 1 at 5:39
YoricYoric
4035
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
add a comment |
If you're comfortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.
You'll need to install it with Homebrew first: brew install rdfind
Then, let's say you want to run a dry test on your ~/Download folder:
rdfind -n true -outputname result.log ~/Downloads/
-n trueis for dry mode: no files will be affected
-outputname result.logwill output the scanning into that file
~/Downloads/is the argument of the root folder you want to scan
(you can pass more than one folder)
Running that command will look something like that:
$ rdfind -n true -outputname result.log ~/Downloads/
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
(DRYRUN MODE) Now have 20132 files in total.
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
(DRYRUN MODE) It seems like you have 15455 files that are not unique
(DRYRUN MODE) Totally, 324 MiB can be reduced.
(DRYRUN MODE) Now making results file result.log
Or, you can use an application with a graphical interface like dupeguru:
answered Jan 1 at 5:39
YoricYoric
4035
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
add a comment |
If you're comfortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.
You'll need to install it with Homebrew first: brew install rdfind
Then, let's say you want to run a dry test on your ~/Download folder:
rdfind -n true -outputname result.log ~/Downloads/
-n trueis for dry mode: no files will be affected
-outputname result.logwill output the scanning into that file
~/Downloads/is the argument of the root folder you want to scan
(you can pass more than one folder)
Running that command will look something like that:
$ rdfind -n true -outputname result.log ~/Downloads/
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
(DRYRUN MODE) Now have 20132 files in total.
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
(DRYRUN MODE) It seems like you have 15455 files that are not unique
(DRYRUN MODE) Totally, 324 MiB can be reduced.
(DRYRUN MODE) Now making results file result.log
Or, you can use an application with a graphical interface like dupeguru:
answered Jan 1 at 5:39
YoricYoric
4035
If you're comfortable using the Terminal, you can try rdfind which purpose is precisely to find duplicate files.
You'll need to install it with Homebrew first: brew install rdfind
Then, let's say you want to run a dry test on your ~/Download folder:
rdfind -n true -outputname result.log ~/Downloads/
-n trueis for dry mode: no files will be affected
-outputname result.logwill output the scanning into that file
~/Downloads/is the argument of the root folder you want to scan
(you can pass more than one folder)
Running that command will look something like that:
$ rdfind -n true -outputname result.log ~/Downloads/
(DRYRUN MODE) Now scanning "/Users/ym/Downloads", found 20132 files.
(DRYRUN MODE) Now have 20132 files in total.
(DRYRUN MODE) Removed 0 files due to nonunique device and inode.
(DRYRUN MODE) Now removing files with zero size from list...removed 75 files
(DRYRUN MODE) Total size is 59782752628 bytes or 56 GiB
(DRYRUN MODE) Now sorting on size:removed 3795 files due to unique sizes from list.16262 files left.
(DRYRUN MODE) Now eliminating candidates based on first bytes:removed 579 files from list.15683 files left.
(DRYRUN MODE) Now eliminating candidates based on last bytes:removed 134 files from list.15549 files left.
(DRYRUN MODE) Now eliminating candidates based on md5 checksum:removed 94 files from list.15455 files left.
(DRYRUN MODE) It seems like you have 15455 files that are not unique
(DRYRUN MODE) Totally, 324 MiB can be reduced.
(DRYRUN MODE) Now making results file result.log
Or, you can use an application with a graphical interface like dupeguru:
answered Jan 1 at 5:39
YoricYoric
4035
edited Jan 1 at 6:47
answered Jan 1 at 5:39
YoricYoric
4035
answered Jan 1 at 5:39
YoricYoric
4035
answered Jan 1 at 5:39
YoricYoric
4035
4035
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
add a comment |
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
Thanks. Awesome!
– AGamePlayer
Jan 1 at 15:25
add a comment |
Thanks for contributing an answer to Ask Different!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fapple.stackexchange.com%2fquestions%2f347217%2ffind-files-with-the-same-content-in-one-folder-and-all-its-sub-folders%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I'd calculate the md5 checksum for all the files in the directory hierarchy and look for checksum duplicates.
– Nimesh Neema
Jan 1 at 5:06