IE11 downloaded files: Files not syncing from client to server because of Temporary attribute

[jochenwezel]

Files downloaded from internet and saved to the owncloud sync folder on a windows system, are not uploaded to the owncloud server.

Background: e.g. IE11 marks downloaded files with following file attributes, marking it as non-trusted:

Bugs in detail

1. Files with this non-trust-level can't be uploaded to server
2. OwnCloud client (Version 2.4.2 (build 10040)) doesn't show any sync errors, but error is expected at this location:
   
3. Files with this non-trust-level in file system are not uploaded after the non-trusting was removed

The owncloud server log doesn't contain any references caused by this issue.

[michaelstingl]

I wasn't able to reproduce with the Win 8.1 (x86) VM with IE11 downloaded from https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/ . @jochenwezel do I need special configuration to see this behavior?

[jochenwezel]

The problem is that I can't analyze the exact error situation for more specific points which might be relevant - because of missing error messages.

Maybe there is some more detail somewhere in owncloud client but hidden, which I could provide if I knew it?

[jochenwezel]

This is how the files appear in Explorer: all files have got the green check mark except the 2 files that don't sync/upload (colouring of file name is because of NTFS file encryption at the whole drive, here it's un-encrypted for the 2 files for testing purposes):

Owncloud client shows status "green":

Restart of owncloud client doesn't change the situation.

[michaelstingl]

Hm, seems those files are also unknown to the explorer.exe extension that draws the overlay icons.

@jochenwezel please start the client with the --logfile and --logdebug parameter. Here you can find more information: https://doc.owncloud.org/desktop/2.5/troubleshooting.html#saving-files-directly

[jochenwezel]

I started the client as following, forced a sync, but the log file keeps at 0 bytes:

C:\Program Files (x86)\ownCloud>owncloud.exe --logfile d:\temp\owncloud-sync.log --logdebug

C:\Program Files (x86)\ownCloud>dir d:\temp\o*
 Datenträger in Laufwerk D: ist Data (unsecured, no-backup)
 Volumeseriennummer: 82CF-04E9

 Verzeichnis von d:\temp

07.08.2018  15:06                 0 owncloud-sync.log
               1 Datei(en),              0 Bytes
               0 Verzeichnis(se), 54.722.663.424 Bytes frei

C:\Program Files (x86)\ownCloud>

After inserting 2 other new files into my owncloud directories, still nothing happens.

Do I have to do any special actions before the owncloud client can log something?

[jochenwezel]

the F12 feature of the owncloud client logs at least 1 line for my sample:

Line 6460: 08-07 16:23:11:314 [ info sync.csync.updater ]:	item: Transfer Temp/Sreenshots_myTemp.pdf - item type 3 not iterated

[ogoffart]

item type 3 not iterated

That's ItemTypeSkip. This can happen if the stat of this file fails, or if this condition is met:

client/src/csync/vio/csync_vio_local_win.cpp

Lines 183 to 186 in 4d8dc13

	} else if (handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_DEVICE
	|| handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_OFFLINE
	|| handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_TEMPORARY) {
	file_stat->type = ItemTypeSkip;

If the stat fails, there should be another log why this happens. And we can't do much anyway in this case.
I would need to so some research for the meaning of the different flag and which one applies in this case. Maybe we should not ignore files with these flags.

(This was introduced in commit 4630244. I don't really know why we exclude the files with these flags.)

[jochenwezel]

Interesting: the attrib command doesn't show any special attributes:

D:\...\OwnCloudSynTestFailures>attrib
A                    D:\...\OwnCloudSynTestFailures\Sreenshots_myTest.pdf
A                    D:\...\OwnCloudSynTestFailures\Test-Flyer.pdf

D:\...\OwnCloudSynTestFailures>attrib /?
Zeigt Dateiattribute an oder ändert sie.

ATTRIB [+R | -R] [+A | -A] [+S | -S] [+H | -H] [+O | -O] [+I | -I] [+X | -X] [+P | -P] [+U | -U]
       [Laufwerk:][Pfad][Dateiname] [/S [/D]] [/L]

  +   Legt ein Attribut fest.
  –   Löscht ein Attribut.
  R   Attribut für "Schreibgeschützte Datei".
  A   Attribut für "Archivdatei".
  S   Attribut für "Systemdatei".
  H   Attribut für "Versteckte Datei".
  O   Attribut für "Offline".
  I   Attribut für "Datei ohne indizierten Inhalt".
  X   Attribut für "Keine Scrubbingdatei".
  V   Attribut für "Integrität".
  P   Attribut für "Angeheftet".
  U   Attribut für "Losgelöst".
  B   Attribut für "SMR-BLOB".
  [Laufwerk:][Pfad][Dateiname]
      Gibt die zu verarbeitenden Dateien für das Attribut an.
  /S  Verarbeitet entsprechende Dateien im aktuellen Ordner
      und allen Unterordnern.
  /D  Verarbeitet auch Ordner.
  /L  Verarbeitet die Attribute des symbolischen Links anstelle
      des Ziels des symbolischen Links.

But file properties dialog in windows shows at least this additional item at one of the files:

I would expect at least a small difference in file attributes at attrib command.

Another test with powershell shows up with some better attributes listing, but still again I'd expect a difference between both files because of the non-trust-level of the first file (see explorer's file property dialog above)

PS D:\...\OwnCloudSynTestFailures> Get-Item *.* | select Name, Mode, Attributes

Name                  Mode                      Attributes
----                  ----                      ----------
Sreenshots_myTest.pdf -a---- Archive, Temporary, Encrypted
Test-Flyer.pdf        -a---- Archive, Temporary, Encrypted

@ogoffart So, it's really the temporary file attribute causing the non-uploading of that file.

At windows platforms, I never realized this temporary file attriute to be used by any windows application (might be different on non-windows platforms).
In general, it would make sense to prevent syncing of temporary files.
Unfortunately, the browser has downloaded the files and saved that attribute in the files - and the user hasn't got a chance to see this hidden attribute with windows standard tools. That's why I recommend to provide a solution at least in the meaning of logging/user guidance so that the user is able to reactivate the sync process for these files.

[jochenwezel]

following powershell statement lists all files which don't sync/upload from client to server:

PS D:\OwnCloudDir> dir -Recurse | ? Attributes -Like *temporary*

[guruz]

This is the same issue as #6610 (comment) :-/ See some research also there.

I think we should have a 2.4.3 that does sync those FILE_ATTRIBUTE_TEMPORARY files?

@jochenwezel Just curious, Is Temporary removed at some point? When you open the file once?

[jochenwezel]

@guruz No, the temporary attribute doesn't change on opening the file - I also don't know any other type of action, that could do the job

The only choice to remove that file attribute is to do some powershell like e.g.

PS D:\...\OwnCloudSynTestFailures> $d = dir
PS D:\...\OwnCloudSynTestFailures> $d[0].Attributes
Archive, Temporary, Encrypted
PS D:\...\OwnCloudSynTestFailures> $d[0].Attributes = 'Archive, Encrypted'
PS D:\...\OwnCloudSynTestFailures> dir | select Name, Attributes

Name                                     Attributes
----                                     ----------
Sreenshots_myTest.pdf            Archive, Encrypted
Test-Flyer.pdf        Archive, Temporary, Encrypted


PS D:\...\OwnCloudSynTestFailures>

Corrected behaviour descriptions

• As soon as the temporary-attribute is removed, the file syncs from the client to the server.
• The no-trust-information shown in the explorer's file settings dialog does NOT influence the syncing (in the beginning of this issue at the very top, this setting was wrongly interpreted as being the cause of this issue)

[jochenwezel]

I think we should have a 2.4.3 that does sync those FILE_ATTRIBUTE_TEMPORARY files?

After reading the issue #6610 and reviewing this issue again, I would follow @guruz and agree to remove that filter for FILE_ATTRIBUTE_TEMPORARY files (if there is no other known reason to keep it as it is), so just

client/src/csync/vio/csync_vio_local_win.cpp

Lines 183 to 186 in 4d8dc13

	} else if (handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_DEVICE
	|| handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_OFFLINE
	|| handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_TEMPORARY) {
	file_stat->type = ItemTypeSkip;

changed to

client/src/csync/vio/csync_vio_local_win.cpp
Lines 183 to 186

 } else if (handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_DEVICE 
             || handle->ffd.dwFileAttributes & FILE_ATTRIBUTE_OFFLINE) { 
     file_stat->type = ItemTypeSkip; 

[guruz]

@jochenwezel Do you want to test with https://download.owncloud.com/desktop/testing/ownCloud-2.4.3.10180rc1-setup.exe ?

[jochenwezel]

@guruz Files with attribute "Temporary" are synced successfully with the provided RC1 exe 👍

[guruz]

Great :) Thanks for testing.