crash on remove account #9367

Talank · 2022-01-17T03:49:34Z

We have seen this error message before, and haven't seen it in a while. However, we need to investigate why it is failing. Since no changes has been merged, and it failed in only one of the nightly build during the weekend, it clearly is a flaky test.

The error details filtered by our log parser is as follows:

{
        "Error Details": "LookupError: Lost connection to AUT",
        "Feature": "remove account connection",
        "Feature File": "tst_removeAccountConnection",
        "Scenario": "remove an account connection",
        "Test Step": "Then an account should be displayed with the displayname Alice Hansen and host %local_server_hostname%"
}

Related drone run (failing): https://drone.owncloud.com/owncloud/client/10363/3/14

individual-it · 2022-01-17T08:54:12Z

from the logs:
02:23:39:023 Info: AUT process '/drone/src/build-GUI-tests/bin/owncloud OWNCLOUD_CORE_DUMP=1 -s --logfile - --logdebug --logflush --confdir /tmp/bdd-tests-owncloud-client/' (PID 5279) finished with exit status -1: Received signal number 11.

TheOneRing · 2022-01-17T10:35:06Z

Yay a crash, shouldn't we get a backtrace?

Talank · 2022-01-19T04:11:12Z

Yup. It was a crash. AUT coredump is still not printed in the log because of #9259. I will look into the fixing apport problem so that this unavailability of AUT crash log does not happen again.

individual-it · 2022-01-20T03:33:31Z

happened again today https://drone.owncloud.com/owncloud/client/10411/3/14

Talank · 2022-02-01T04:05:44Z

Failed again today https://drone.owncloud.com/owncloud/client/10514/3/14

individual-it · 2022-02-01T04:17:33Z

@Talank Any idea what happens here and what we could do?

Talank · 2022-02-01T04:24:20Z

Well, we do have the squish backtrace in the log. But clearly it is an AUT crash causing the connection lost error. We can start investigating from the log that we already have, but I highly doubt that there will be any other outcomes from that investigation. The backtracking works perfectly in local test runs so we can try to reproduce it locally, which might give us some additional idea (if we can reproduce it).

Talank · 2022-02-04T17:41:41Z

@TheOneRing I reproduced the issue locally. Here is the full backtrace
backtrace.txt

TheOneRing · 2022-02-21T08:11:28Z

The crash seems to occur in line 166.

client/src/gui/activitywidget.cpp

Lines 164 to 168 in 105d39c

    
           void ActivityWidget::slotAccountActivityStatus(AccountState *ast, int statusCode) 
        
           { 
        
               if (!(ast && ast->account())) { 
        
                   return; 
        
               }

And then when accessing the shared ptr

client/src/gui/accountstate.cpp

Line 145 in 105d39c

AccountPtr AccountState::account() const

individual-it · 2022-02-21T08:48:36Z

@TheOneRing Talank took off the QA-team label. Please reach out if you need more help here

TheOneRing · 2022-02-28T14:09:27Z

@Talank can you pls describe the necessary steps?

Fixes: #9367

Talank · 2022-03-01T07:33:12Z

@TheOneRing I reproduced the crash by running the automated test remove an account connection several times (about 20 times). The steps in the scenario is as follows:

    - Create an user "Alice"
    - Create an user "Brian"
    - set up client for user "Alice" with default settings
    - add user "Brian" to client, also with default settings
    - remove the connection for user "Brian"
    - Then "Brain" should be removed and "Alice" should not be affected

Fixes: #9367

TheOneRing · 2022-03-01T08:53:03Z

A change was merged and the issue is hopefully fixed.

Talank · 2022-03-02T04:38:41Z

I ran the test (more than 20 times) that reproduced the crash. And I am not being able to reproduce it anymore in 2.10 branch. So I agree that the issue is hopefully fixed.

TheOneRing · 2022-03-02T07:31:36Z

Thx

Talank · 2022-03-19T03:32:52Z

I noticed the crash again in #9518 (comment) and in #9525 (comment)

fmoc · 2022-03-21T11:22:42Z

I just tried to reproduce the issue and capture it with a debugger following the steps listed in #9367 (comment). In over 40 repetitions, I couldn't produce a crash, thus I cannot tell where the crash occurs. How should we proceed with this issue?

Talank · 2022-03-21T11:42:20Z

Earlier, in #9367 (comment) I reproduced it by running the automated test (that captured the crash) more than 20 times continuously.

fmoc · 2022-03-31T13:58:11Z

We now have stacktraces from the CI builds, see #9555 (comment). The stacktrace linked from there:

###########################################
Scenario: remove an account connection
-------------------------------------------
Executable: /drone/src/build-GUI-tests/bin/owncloud
Timestamp: 2022-03-31 06:50:20
Process ID: 3328
Signal Number: 6
-------------------------------------------
<<<<< STACKTRACE START >>>>>
[New LWP 3328]
[New LWP 3330]
[New LWP 3344]
[New LWP 3341]
[New LWP 3349]
[New LWP 3331]
[New LWP 3332]
[New LWP 3347]
[New LWP 3354]
[New LWP 3333]
[New LWP 3361]
[New LWP 3370]
[New LWP 3343]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/drone/src/build-GUI-tests/bin/owncloud OWNCLOUD_CORE_DUMP=1 -s --logfile - --l'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
[Current thread is 1 (Thread 0x7efddbd83900 (LWP 3328))]
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {1024, 0 <repeats 11 times>, 140735172216104, 139628831405824, 93952587721792, 93952590533136}}
        pid = <optimized out>
        tid = <optimized out>
#1  0x00007efdde958859 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x7fff75f287d0, sa_sigaction = 0x7fff75f287d0}, sa_mask = {__val = {177551552, 140735172216784, 139628748260981, 0, 139628826644155, 140735172216816, 140735172216920, 140735172216864, 0, 139628840639240, 140735172219184, 139628840639240, 10474322719258358784, 93952593341424, 140735172216920, 140735172218432}}, sa_flags = 0, sa_restorer = 0x7fff75f28858}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007efdd97076a8 in crashHandler () at /home/headless/squish/lib/libsquishhook.so
#3  0x00007efdde9790c0 in <signal handler called> () at /lib/x86_64-linux-gnu/libc.so.6
#4  0x0000557308c3155b in std::__atomic_base<int>::operator++() (this=0x7efddf545680 <QArrayData::shared_null>) at /usr/include/c++/9/bits/atomic_base.h:319
#5  0x0000557308c2f836 in QAtomicOps<int>::ref<int>(std::atomic<int>&) (_q_value=...) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qatomic_cxx11.h:265
#6  0x0000557308c2d6cc in QBasicAtomicInteger<int>::ref() (this=0x7efddf545680 <QArrayData::shared_null>) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qbasicatomic.h:114
#7  0x0000557308c45bde in QSharedPointer<OCC::Account>::ref() const (this=0x7fff75f28f20) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qsharedpointer_impl.h:515
#8  0x0000557308c431fd in QSharedPointer<OCC::Account>::QSharedPointer(QSharedPointer<OCC::Account> const&) (this=0x7fff75f28f20, other=...) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qsharedpointer_impl.h:331
#9  0x0000557308ca8443 in OCC::AccountState::account() const (this=0x7efd9c0009e0) at ../src/gui/accountstate.cpp:151
#10 0x0000557308d4f081 in OCC::ActivityWidget::slotAccountActivityStatus(OCC::AccountState*, int) (this=0x557309ba88c0, ast=0x7efd9c0009e0, statusCode=999) at ../src/gui/activitywidget.cpp:170
#11 0x0000557308d5973f in QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<OCC::AccountState*, int>, void, void (OCC::ActivityWidget::*)(OCC::AccountState*, int)>::call(void (OCC::ActivityWidget::*)(OCC::AccountState*, int), OCC::ActivityWidget*, void**) (f=(void (OCC::ActivityWidget::*)(class OCC::ActivityWidget * const, class OCC::AccountState *, int)) 0x557308d4f038 <OCC::ActivityWidget::slotAccountActivityStatus(OCC::AccountState*, int)>, o=0x557309ba88c0, arg=0x7fff75f29130) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:152
#12 0x0000557308d58e41 in QtPrivate::FunctionPointer<void (OCC::ActivityWidget::*)(OCC::AccountState*, int)>::call<QtPrivate::List<OCC::AccountState*, int>, void>(void (OCC::ActivityWidget::*)(OCC::AccountState*, int), OCC::ActivityWidget*, void**) (f=(void (OCC::ActivityWidget::*)(class OCC::ActivityWidget * const, class OCC::AccountState *, int)) 0x557308d4f038 <OCC::ActivityWidget::slotAccountActivityStatus(OCC::AccountState*, int)>, o=0x557309ba88c0, arg=0x7fff75f29130) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:185
#13 0x0000557308d57751 in QtPrivate::QSlotObject<void (OCC::ActivityWidget::*)(OCC::AccountState*, int), QtPrivate::List<OCC::AccountState*, int>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (which=1, this_=0x557309bce430, r=0x557309ba88c0, a=0x7fff75f29130, ret=0x0) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:414
#14 0x00007efddf490328 in QMetaObject::activate(QObject*, int, int, void**) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#15 0x0000557308ce3bef in OCC::ActivityListModel::activityJobStatusCode(OCC::AccountState*, int) (this=0x557309bd7630, _t1=0x7efd9c0009e0, _t2=999) at src/gui/owncloudCore_autogen/IVK4DJNV27/moc_activitylistmodel.cpp:167
        _a = {0x0, 0x7fff75f29120, 0x7fff75f2911c}
#16 0x0000557308d7444e in OCC::ActivityListModel::<lambda(const QJsonDocument&, int)>::operator()(const QJsonDocument &, int) const (__closure=0x55730aa03eb0, json=..., statusCode=0) at ../src/gui/models/activitylistmodel.cpp:203
        activities = {d = 0x0, a = 0x0}
        list = {<QListSpecialMethods<OCC::Activity>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x55730a9dc710}, d = 0x55730a9dc710}}
        this = 0x557309bd7630
        ast = 0x7efd9c0009e0
        job = 0x557309cefe50
        job = 0x55730a8d6b00
        params = {d = {d = 0x0}}
        __PRETTY_FUNCTION__ = "void OCC::ActivityListModel::startFetchJob(OCC::AccountState*)"
#17 0x0000557308d75729 in QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<const QJsonDocument&, int>, void, OCC::ActivityListModel::startFetchJob(OCC::AccountState*)::<lambda(const QJsonDocument&, int)> >::call(OCC::ActivityListModel::<lambda(const QJsonDocument&, int)> &, void **) (f=..., arg=0x7fff75f29590) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:146
#18 0x0000557308d756e0 in QtPrivate::Functor<OCC::ActivityListModel::startFetchJob(OCC::AccountState*)::<lambda(const QJsonDocument&, int)>, 2>::call<QtPrivate::List<QJsonDocument const&, int>, void>(OCC::ActivityListModel::<lambda(const QJsonDocument&, int)> &, void *, void **) (f=..., arg=0x7fff75f29590) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:256
#19 0x0000557308d756ae in QtPrivate::QFunctorSlotObject<OCC::ActivityListModel::startFetchJob(OCC::AccountState*)::<lambda(const QJsonDocument&, int)>, 2, QtPrivate::List<const QJsonDocument&, int>, void>::impl(int, QtPrivate::QSlotObjectBase *, QObject *, void **, bool *) (which=1, this_=0x55730aa03ea0, r=0x557309bd7630, a=0x7fff75f29590, ret=0x0) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:439
#20 0x00007efddf490328 in QMetaObject::activate(QObject*, int, int, void**) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#21 0x00007efddfa95e2f in OCC::JsonApiJob::jsonReceived(QJsonDocument const&, int) (this=0x557309cefe50, _t1=..., _t2=0) at src/libsync/libsync_autogen/EWIEGA46WW/moc_networkjobs.cpp:1486
        _a = {0x0, 0x7fff75f29610, 0x7fff75f2957c}
#22 0x00007efddfafdc41 in OCC::JsonApiJob::finished() (this=0x557309cefe50) at ../src/libsync/networkjobs.cpp:782
        __PRETTY_FUNCTION__ = "virtual bool OCC::JsonApiJob::finished()"
        statusCode = 0
        jsonStr = {static null = {<No data fields>}, d = 0x55730a957550}
        error = {offset = 500, error = 21875}
        json = {static BinaryFormatTag = 1936351857, d = 0x55730a957550}
#23 0x00007efddfaf3b87 in OCC::AbstractNetworkJob::slotFinished() (this=0x557309cefe50) at ../src/libsync/abstractnetworkjob.cpp:276
        __PRETTY_FUNCTION__ = "void OCC::AbstractNetworkJob::slotFinished()"
        maxHttp2Resends = 3
        verb = {d = 0x7efddfb8d600 <OCC::HttpLogger::requestVerb(QNetworkAccessManager::Operation, QNetworkRequest const&)::{lambda()#2}::operator()() const::qbytearray_literal>}
        discard = 117
#24 0x00007efddfaf6ce9 in QtPrivate::FunctorCall<QtPrivate::IndexesList<>, QtPrivate::List<>, void, void (OCC::AbstractNetworkJob::*)()>::call(void (OCC::AbstractNetworkJob::*)(), OCC::AbstractNetworkJob*, void**) (f=(void (OCC::AbstractNetworkJob::*)(class OCC::AbstractNetworkJob * const)) 0x7efddfaf289e <OCC::AbstractNetworkJob::slotFinished()>, o=0x557309cefe50, arg=0x7fff75f29890) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:152
#25 0x00007efddfaf6bdc in QtPrivate::FunctionPointer<void (OCC::AbstractNetworkJob::*)()>::call<QtPrivate::List<>, void>(void (OCC::AbstractNetworkJob::*)(), OCC::AbstractNetworkJob*, void**) (f=(void (OCC::AbstractNetworkJob::*)(class OCC::AbstractNetworkJob * const)) 0x7efddfaf289e <OCC::AbstractNetworkJob::slotFinished()>, o=0x557309cefe50, arg=0x7fff75f29890) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:185
#26 0x00007efddfaf6a7b in QtPrivate::QSlotObject<void (OCC::AbstractNetworkJob::*)(), QtPrivate::List<>, void>::impl(int, QtPrivate::QSlotObjectBase*, QObject*, void**, bool*) (which=1, this_=0x55730a96d310, r=0x557309cefe50, a=0x7fff75f29890, ret=0x0) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:414
#27 0x00007efddf490328 in QMetaObject::activate(QObject*, int, int, void**) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#28 0x00007efddf92072c in  () at /lib/x86_64-linux-gnu/libQt5Network.so.5
#29 0x00007efddf9d17ce in  () at /lib/x86_64-linux-gnu/libQt5Network.so.5
#30 0x00007efddf490c2a in QObject::event(QEvent*) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#31 0x00007efde038ca66 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#32 0x00007efde03960f0 in QApplication::notify(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#33 0x00007efddf46480a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#34 0x00007efddf467488 in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#35 0x00007efddf4bce37 in  () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#36 0x00007efddeeab17d in g_main_context_dispatch () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#37 0x00007efddeeab400 in  () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#38 0x00007efddeeab4a3 in g_main_context_iteration () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#39 0x00007efddf4bc435 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#40 0x00007efddf4633ab in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#41 0x00007efddf46b116 in QCoreApplication::exec() () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#42 0x0000557308c23c6a in main(int, char**) (argc=9, argv=0x7fff75f2a218) at ../src/gui/main.cpp:157
        app = {<SharedTools::QtSingleApplication> = {<QApplication> = {<No data fields>}, static staticMetaObject = {d = {superdata = 0x7efde088d620 <QApplication::staticMetaObject>, stringdata = 0x557308eb3360 <qt_meta_stringdata_SharedTools__QtSingleApplication>, data = 0x557308eb3500 <qt_meta_data_SharedTools__QtSingleApplication>, static_metacall = 0x557308cdc62c <SharedTools::QtSingleApplication::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}, firstPeer = -1, instances = 0x557309aafe90, pidPeer = 0x557309ab74f0, actWin = 0x0, appId = {static null = {<No data fields>}, d = 0x7efddfb9cbc0 <OCC::Theme::appName() const::{lambda()#1}::operator()() const::qstring_literal>}, block = false}, static staticMetaObject = {d = {superdata = 0x557308f354a0 <SharedTools::QtSingleApplication::staticMetaObject>, stringdata = 0x557308eb5600 <qt_meta_stringdata_OCC__Application>, data = 0x557308eb5a20 <qt_meta_data_OCC__Application>, static_metacall = 0x557308cdeb68 <OCC::Application::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}, _gui = {wp = {d = 0x557309ccac00, value = 0x557309ba5b40}}, _theme = 0x557309a60a60, _showSettings = true, _quitInstance = false, _logFile = {static null = {<No data fields>}, d = 0x557309ad2e40}, _logDir = {static null = {<No data fields>}, d = 0x7efddf545680 <QArrayData::shared_null>}, _logExpire = {__r = 0}, _logFlush = true, _logDebug = true, _userTriggeredConnect = false, _debugMode = false, _userEnforcedLanguage = {static null = {<No data fields>}, d = 0x7efddf545680 <QArrayData::shared_null>}, _displayLanguage = {static null = {<No data fields>}, d = 0x557309b093f0}, _proxy = {<QObject> = {<No data fields>}, static staticMetaObject = {d = {superdata = 0x7efddf716980 <QObject::staticMetaObject>, stringdata = 0x557308eb5d00 <qt_meta_stringdata_OCC__ClientProxy>, data = 0x557308eb5d80 <qt_meta_data_OCC__ClientProxy>, static_metacall = 0x557308cdf1e8 <OCC::ClientProxy::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}}, _networkConfigurationManager = <incomplete type>, _checkConnectionTimer = <incomplete type>, _folderManager = {d = 0x557309b0d4d0}}
        __PRETTY_FUNCTION__ = "int main(int, char**)"
        updater = 0x557309c8ec60

<<<<< STACKTRACE END >>>>>
###########################################

The AccountPtr seems to be broken, perhaps due to premature cleanup of its members. I'm not sure how that could happen, though. The atomic int used for ref counting seems to be dangling. Seems like the activity widget tries to access the (no longer available) account object, but that makes little sense given the existing code in slotAccountActivityStatus...

individual-it · 2022-05-16T04:07:24Z

happened again on 14.05
Stacktrace: https://cache.owncloud.com/public/owncloud/client/11763/guiReportUpload/stacktrace.log

TheOneRing · 2022-05-18T14:44:20Z

Please retest on master, we don't expect another 2.10 release

TheOneRing · 2022-06-29T10:42:51Z

fixed on master

Talank added the QA:team label Jan 17, 2022

Talank added the p2-high Escalation, on top of current planning, release blocker label Feb 1, 2022

TheOneRing added this to the 2.10.1 milestone Feb 21, 2022

Talank removed the QA:team label Feb 21, 2022

TheOneRing changed the title ~~remove an account connection scenario fails~~ crash on remove account Feb 23, 2022

TheOneRing added a commit that referenced this issue Feb 28, 2022

Fix shared pointer deletion before the slot was invoked

e78d52c

Fixes: #9367

TheOneRing mentioned this issue Feb 28, 2022

Fix shared pointer deletion before the slot was invoked #9475

Merged

TheOneRing added a commit that referenced this issue Feb 28, 2022

Fix shared pointer deletion before the slot was invoked

c13b23b

Fixes: #9367

TheOneRing added a commit that referenced this issue Mar 1, 2022

Fix shared pointer deletion before the slot was invoked

83d238f

Fixes: #9367

TheOneRing closed this as completed Mar 1, 2022

TheOneRing mentioned this issue Mar 10, 2022

Release Desktop 2.10.1 #9488

Closed

55 tasks

Talank mentioned this issue Mar 18, 2022

Retry update after 10 minutes #9525

Merged

Talank reopened this Mar 21, 2022

This comment was marked as off-topic.

Sign in to view

fmoc modified the milestones: 2.10.1, 2.10 Mar 30, 2022

Talank mentioned this issue May 10, 2022

GUI tests failed with AUT crash #9650

Closed

Talank mentioned this issue May 18, 2022

[Tests-Only] Fixed tests related to expiration date set/change #9463

Merged

TheOneRing modified the milestones: 2.10, 3.0 May 18, 2022

TheOneRing closed this as completed Jun 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crash on remove account #9367

crash on remove account #9367

Talank commented Jan 17, 2022 •

edited

Loading

individual-it commented Jan 17, 2022

TheOneRing commented Jan 17, 2022

Talank commented Jan 19, 2022

individual-it commented Jan 20, 2022

Talank commented Feb 1, 2022

individual-it commented Feb 1, 2022

Talank commented Feb 1, 2022

Talank commented Feb 4, 2022 •

edited

Loading

TheOneRing commented Feb 21, 2022 •

edited

Loading

individual-it commented Feb 21, 2022 •

edited

Loading

TheOneRing commented Feb 28, 2022

Talank commented Mar 1, 2022 •

edited

Loading

TheOneRing commented Mar 1, 2022

Talank commented Mar 2, 2022

TheOneRing commented Mar 2, 2022

Talank commented Mar 19, 2022 •

edited

Loading

fmoc commented Mar 21, 2022

Talank commented Mar 21, 2022

This comment was marked as off-topic.

fmoc commented Mar 31, 2022 •

edited

Loading

individual-it commented May 16, 2022

TheOneRing commented May 18, 2022

TheOneRing commented Jun 29, 2022

crash on remove account #9367

crash on remove account #9367

Comments

Talank commented Jan 17, 2022 • edited Loading

individual-it commented Jan 17, 2022

TheOneRing commented Jan 17, 2022

Talank commented Jan 19, 2022

individual-it commented Jan 20, 2022

Talank commented Feb 1, 2022

individual-it commented Feb 1, 2022

Talank commented Feb 1, 2022

Talank commented Feb 4, 2022 • edited Loading

TheOneRing commented Feb 21, 2022 • edited Loading

individual-it commented Feb 21, 2022 • edited Loading

TheOneRing commented Feb 28, 2022

Talank commented Mar 1, 2022 • edited Loading

TheOneRing commented Mar 1, 2022

Talank commented Mar 2, 2022

TheOneRing commented Mar 2, 2022

Talank commented Mar 19, 2022 • edited Loading

fmoc commented Mar 21, 2022

Talank commented Mar 21, 2022

This comment was marked as off-topic.

fmoc commented Mar 31, 2022 • edited Loading

individual-it commented May 16, 2022

TheOneRing commented May 18, 2022

TheOneRing commented Jun 29, 2022

Talank commented Jan 17, 2022 •

edited

Loading

Talank commented Feb 4, 2022 •

edited

Loading

TheOneRing commented Feb 21, 2022 •

edited

Loading

individual-it commented Feb 21, 2022 •

edited

Loading

Talank commented Mar 1, 2022 •

edited

Loading

Talank commented Mar 19, 2022 •

edited

Loading

fmoc commented Mar 31, 2022 •

edited

Loading