[Bug]: 'Data directory protected' check failing #45087
Comments
nmbgeek
added
0. Needs triage
|
Modified function in DataDirectoryProtected.php: Error logs this: Not sure why it doesn't get my webRoot directory though or why the |
Unless you've deployed Nextcloud in a subfolder, webroot is expected to be empty (well, or What happens if you try to access the full URLs being used in the check by utilizing each of your configured https://domain1.com/var/nc_data/.ocdata Or, if you prefer: |
|
That would be the problem. One of my trusted domains was the box's internal FQDN but the box's DNS is set to public servers. Is there any reason for not bypassing the check if the data directory isn't inside of the webroot? A check like I created in this possibly: https://github.com/nmbgeek/nextcloud-server/blob/master/apps/settings/lib/SetupChecks/DataDirectoryProtected.php or at least have it only check the value in 'overwrite.cli.url'? |
That shouldn't be a problem. We already anticipate that scenario; a similar check is used for You didn't answer my question: what happens if you try to access each of the full URLs being used in the check? Does one of them succeed?
Symbolic links come to mind (maybe? I haven't had my morning coffee yet). But it shouldn't matter either way, the test should pass unless we've either got a bug or an
It doesn't matter if some of the |
The second domain resolves to a different webserver on public DNS due to wildcard subdomain dns record which returns a 200 status code for its generic default page. Internal DNS resolves it directly rather than having to hairpin at my router which then forwards it to Nginx Proxy Manager before hitting the nextcloud server. The nextcloud server's DNS is set to public DNS rather than internal so it is resolving the domain to a public address unlike clients using internal dns which resolve it directly to the nextcloud box. It was setup some time ago like that for some obscure reason, I believe it had something to do with troubleshooting NPM, and I have removed that domain so the "issue" I was having of it reporting the erroneous error is technically resolved. I think the check could be done better as it never was the issue reported by nextcloud that my .htaccess wasn't working and that my data directory was possibly publicly available. |
|
I can confirm that removing the network's central proxy server (also Nginx Proxy Manager) from the trusted domains in Nextcloud's config resolved the issue. And I also forgot why I had to add the proxy server in the first place. |
|
For me neither works. Everything in the config is correct but still for some reason the error is there. i have checked several times that the data directory is not accessible. folder is even deleted. Still the check fails. On the current system where it shows we only have one trusted domain. overwrite is correct. |
|
curl -I https://.ddns.net/mnt/ncdata/.ocdata |
|
This is strange. Curl gives a 404 but the code it self suggest that its getting a 200 ok. |
|
I think there is actually an improvement possible for the implementation of the check: If your request is redirected, and you don't receive |
|
Posting this here as well nextcloud/docker#2214 (comment) since I think it's relevant for this issue. |
|
What about if the check was for the .ocdata file with a non-forbidden status code and a content-length of 0? Since the .ocdata file is just an empty file if it is actually getting the real ocdata file it should have a content length of 0. |
|
I have been testing around more. I added debug to the actual check itself to se what url and what host it checks. On this system it is only one host. And the complete url that it checks gives a 404 with curl. |
|
Found it! I dont know why i missed this. Rewrite rule on port 80 for https generates a 301 as response. That is why its failing. |
So, the Nextcloud PHP code should be able to handle redirects. That's the solution. |
Or correctly generate Test-URLs honoring |
Good point! |
|
it tests for missconfiguraton. Imagine if you forgot to protect stuff on port 80 but had a perfect config for ssl. |
Also good point! 😆 |
I have only set trusted_domains to the domain nextcloud is running (no trusted_proxy) and I can confirm that once I remove 80 -> 443 redirect, the error is finally gone! \o/ |
|
you can add a redirect without generationg a 301 and it would work. Either a better rewrite rule or a Redirct statement |
I'm not following why you would want to disable https redirects though.
The default certbot rule that it creates for port 80 is below. Why wouldn't you want all http traffic redirected to https? |
|
Hi. I also have the same issue but using nginx directly, not Apache. The test is failing, even though the data directory returns a 404, by design. It exists in the webroot, but nginx is configured to hide it by default: I'm confused as to why the .htaccess file is tested, since this will always result in a false negative when using nginx. I know nginx isn't officially supported by Nextcloud, but you can't take for granted in the code that Apache is used. If a request to .ocdata returns anything other that 200, the test shouldn't fail, right ? In my case a request to anything under /data does return 404 as expected, so the test should be valid since the data isn't accessible. |
The .htaccess itself isn't tested. The current design does a curl for the location of your data directory appended to all of your trusted_domains from your config. Review your config file for trusted domains and where it locates your data directory as being (minus the server root portion of your data directory if applicable). Then append /.ocdata to that and perform a 'curl -l' from your server's cli. The actual check can be found here: https://github.com/nextcloud/server/blob/master/apps/settings/lib/SetupChecks/DataDirectoryProtected.php I just think this check could be done much better and be made to be more robust. |
|
@enoch85 after running the "Activate TLS" script this Nextcloud warning is not gone away. Its saying that my data directory might be accessible from the internet and .htaccess file is not working. But as before no really helpful messages what test has failed or more information to really solve this kind of warning. Error Message in Nextcloud: And doing the curl command as before the call to |
Please continue in nextcloud/vm#2645 as this notifies everyone in this issue. |
|
I am also getting this warning message after upgrading from 28.x to 29.0.2 This is on a Hansson IT VM which has never had the data directory inside the webroot. The message is confusing, and induces an unjustified fear that there is a threat due to a misconfiguration. In my case, checking the server via https://scan.nextcloud.com/ results in an A+ rating so this is a bug that should be resolved by changing the code, and not require users to tinker with settings that were perfectly ok up to Nextcloud version 28. And no, re-running the TLS enable script did not work. |
|
@anschluss-org we are looking in to that in the vm. What is failing is a redirect. if you look in this thread there are working samples that will fix it. |
That's good to hear. |
so we´ll wait a while |
This is now fixed. Please re-run the Activate TLS script. |
my nextcloud was installed with a dedyn.io-address, so how do I run the Activate TLS script, with which url? on my router port 80 is not open.... |
Just run the dedyn script again. :) |
ok, I run the sript again, I created a new token and the desec domain was successful created, but when the script creates a new certificate, it runs in some errors: Generating new TLS cert with DNS and deSEC, please don't abort the hook, it may take a while... Certbot failed to authenticate some domains (authenticator: manual). The Certificate Authority reported these problems: Hint: The Certificate Authority failed to verify the DNS TXT records created by the --manual-auth-hook. Ensure that this hook is functioning correctly and that it waits a sufficient duration of time for DNS propagation. Refer to "certbot --help manual" and the Certbot User Guide. Hook '--manual-cleanup-hook' for xxxxx.dedyn.io ran with output: |
Please send an email to support at hansson it.se |
|
After NC 28.* upgrade to 29.*.latest , get this "Your data directory and files are probably accessible from the internet" (sounds too dangerous for me) and can't resolve it, shutdown my self-hosted Nextcloud instance for now. Let's Encrypt wildcard cert that works for https://nextcloud.example.com, latest Debian, PHP FPM 8.3, data in I understand, you do your best, but I think, that /apps/settings/lib/SetupChecks/DataDirectoryProtected.php do something wrong |
If your data isn't in the webroot you should have nothing to be concerned about unless your server is otherwise compromised. The data directory protected task is doing a simple curl with your data directory appended to your webroot. So curl from your cli to https://nextcloud.example.com/data/nextcloud (root domain + the /data/nextcloud path you provided) along with any other domains you have in the config. If any of those give a redirect instead of a forbidden this check fails. I don't believe there is any reason to take your NC offline and that is why this issue is open because this check fails when the data directory is clearly protected and never even actually checks your .htaccess or access to any actual data. |
|
multiple check everything, but same "Your data directory and files are probably accessible from the internet"
|
|
I am using Caddy web server, which does not use |
|
Yeah I've had a similar issue when an IP and "localhost" were inside my trusted domains. I guess the checker tried to connect via HTTPS and the certificate validation failed, so no 404 was returned? |
|
I have valid LE wildcard certificate, trusted _domain is one, FQDN but your version, that my system don't just return 404, but something like blue Nextcloud page with "this and this not found" sounds reasonable |
|
I just realize, that it may be related with "Hosts and FQDN" configuration issues (even my Nextcloud instance isn't snap): https://github.com/nextcloud-snap/nextcloud-snap/wiki/Hosts-and-FQDN-for-Nextcloud-snap hostnamectl gives internal64-4.example.com (in my LAN FQDN) but in reality it also acts as FQDNs:
|
|
This help me: After: Where I have proxy (Apache mod_proxy) with SSL termination and DMZ with plain http from proxy to server (I don't show this part of the config in this message). |
|
Issue still present in 29.0.5 |
And still on 29.0.6 getting a false positive on older (~2 year old) installation(s) |
|
after update to: Nextcloud Hub 8 (29.0.7) |
|
Can confirm as well that 29.0.7 fixed it ! |
|
Not resolved for me with 29.0.7 (running from CLI with After digging a bit into the code, I checked the logs for The web server logs show exactly one request to However, if I add some logging to |
|
Me too I am struggling to find out why some tests fail. should rather be ??? This bug will lead to some URLs not being checked, that should be checked... And in the same function, should rather be (removeWebroot always true)? Because cliUrl already contains the webroot... |
|
I've just made this error disappear on my installation by removing the installation's old domain name from Hope this helps! |
and others
Bug description
The Data directory protected check DataDirectoryProtected.php still appears to try and check the URL even when the data directory is not in the webroot. The error also indicated that the .htaccess file is not working however it is. The runHead function is being passed $dataUrl which when I modified the code to log $dataUrl showed as just
//var/nc_dataand the .oc_data was also not appended.Steps to reproduce
Expected behavior
The check should pass if data directory is not inside of the webroot.
Installation method
Community Manual installation with Archive
Nextcloud Server version
29
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.3
Web server
Apache (supported)
Database engine version
MySQL
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 22 to 23)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
{ "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ ***Manually Removed*** ], "trusted_proxies": "***REMOVED SENSITIVE VALUE***", "datadirectory": "/var/nc_data", "dbtype": "mysql", "version": "29.0.0.19", "overwrite.cli.url": "***Manually Removed***", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "activity_expire_days": 14, "auth.bruteforce.protection.enabled": "true", "simpleSignUpLink.shown": false, "blacklisted_files": [ ".htaccess", "Thumbs.db", "thumbs.db" ], "cron_log": true, "default_phone_region": "US", "enable_previews": true, "enabledPreviewProviders": [ "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap", "OC\\Preview\\Movie", "OC\\Preview\\PDF", "OC\\Preview\\MP3", "OC\\Preview\\TXT", "OC\\Preview\\MarkDown" ], "filesystem_check_changes": 0, "filelocking.enabled": true, "htaccess.RewriteBase": "\/", "integrity.check.disabled": false, "knowledgebaseenabled": false, "log_type": "file", "logfile": "\/var\/nc_data\/nextcloud.log", "loglevel": 2, "logdateformate": "F d, Y H:i:s", "logtimezone": "America\/New_York", "log_rotate_size": 104857600, "maintenance": false, "memcache.local": "\\OC\\Memcache\\Redis", "memcache.locking": "\\OC\\Memcache\\Redis", "overwriteprotocol": "https", "preview_max_x": 16000, "preview_max_y": 16000, "preview_max_scale_factor": 1, "preview_max_memory": 512, "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 0, "timeout": 0 }, "quota_include_external_storage": false, "share_folder": "\/Shares", "skeletondirectory": "", "theme": "", "trashbin_retention_obligation": "auto, 7", "updater.release.channel": "stable", "mail_smtpmode": "smtp", "mail_smtpsecure": "ssl", "mail_sendmailmode": "smtp", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtpauthtype": "LOGIN", "mail_smtpauth": 1, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "465", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "maintenance_window_start": 5, "app_install_overwrite": [ "files_automatedtagging", "imageconverter", "files_archive" ] } }List of activated Apps
Nextcloud Signing status
Nextcloud Logs
{"reqId":"OCPRKEMAJw49JilEK9oK","level":2,"time":"2024-04-28T19:51:20-04:00","remoteAddr":"10.0.0.70","user":"user-omitted","app":"no app in context","method":"GET","url":"/core/preview?fileId=604398&x=32&y=32&mimeFallback=true&a=0","message":"Transaction took 1.2166879177094s","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36","version":"29.0.0.19","exception":{"Exception":"Exception","Message":"Transaction took 1.2166879177094s","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/DB/ConnectionAdapter.php","line":154,"function":"commit","class":"OC\\DB\\Connection","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":523,"function":"commit","class":"OC\\DB\\ConnectionAdapter","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":404,"function":"handleChildren","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":354,"function":"scanChildren","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/LocalRootScanner.php","line":39,"function":"scan","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1341,"function":"scan","class":"OC\\Files\\Cache\\LocalRootScanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":1380,"function":"getCacheEntry","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/Root.php","line":208,"function":"getFileInfo","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Node/LazyFolder.php","line":161,"function":"get","class":"OC\\Files\\Node\\Root","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/AppData/AppData.php","line":98,"function":"get","class":"OC\\Files\\Node\\LazyFolder","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/AppData/AppData.php","line":147,"function":"getAppDataFolder","class":"OC\\Files\\AppData\\AppData","type":"->"},{"file":"/var/www/nextcloud/lib/private/Preview/Storage/Root.php","line":74,"function":"newFolder","class":"OC\\Files\\AppData\\AppData","type":"->"},{"file":"/var/www/nextcloud/lib/private/Preview/Generator.php","line":607,"function":"newFolder","class":"OC\\Preview\\Storage\\Root","type":"->"},{"file":"/var/www/nextcloud/lib/private/Preview/Generator.php","line":133,"function":"getPreviewFolder","class":"OC\\Preview\\Generator","type":"->"},{"file":"/var/www/nextcloud/lib/private/Preview/Generator.php","line":110,"function":"generatePreviews","class":"OC\\Preview\\Generator","type":"->"},{"file":"/var/www/nextcloud/lib/private/PreviewManager.php","line":187,"function":"getPreview","class":"OC\\Preview\\Generator","type":"->"},{"file":"/var/www/nextcloud/core/Controller/PreviewController.php","line":174,"function":"getPreview","class":"OC\\PreviewManager","type":"->"},{"file":"/var/www/nextcloud/core/Controller/PreviewController.php","line":142,"function":"fetchPreview","class":"OC\\Core\\Controller\\PreviewController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":232,"function":"getPreviewByFileId","class":"OC\\Core\\Controller\\PreviewController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":338,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1050,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/lib/private/DB/Connection.php","Line":691,"message":"Transaction took 1.2166879177094s","exception":{},"CustomMessage":"Transaction took 1.2166879177094s"}}Additional info
The errors in logs appear when browsing various file heavy folders and/or the Photos app.
The text was updated successfully, but these errors were encountered: