Bugs on stable in map creation #1031

jywarren · 2019-09-16T14:10:52Z

Follow-up to #1028 -- seeing:

I, [2019-09-16T14:07:27.930936 #233]  INFO -- : [f4b21fe7-7c7b-4a07-abb3-3d
4a05ed6546] Completed 500 Internal Server Error in 5ms (ActiveRecord: 0.9ms
)
D, [2019-09-16T14:07:27.931284 #233] DEBUG -- : [f4b21fe7-7c7b-4a07-abb3-3d
4a05ed6546] cannot load such file -- aws-sdk-s3 (You may need to install th
e aws-sdk-s3 gem) excluded from capture: DSN not set
4a05ed6546] app/controllers/images_controller.rb:28:in `create'

I had an error on uploading an image -- Error Internal Server Error in the modal -

And one on submitting the create map form while anonymous -- probably a 500.

The text was updated successfully, but these errors were encountered:

jywarren · 2019-09-16T14:11:30Z

2nd error (map creation when not logged in) is:

7459225be3] Completed 500 Internal Server Error in 33ms (ActiveRecord: 3.9m
s)
D, [2019-09-16T14:07:02.111838 #233] DEBUG -- : [b08fd659-a7f0-4a66-b203-5e
7459225be3] undefined method `zero?' for nil:NilClass excluded from capture
: DSN not set
F, [2019-09-16T14:07:02.113698 #233] FATAL -- : [b08fd659-a7f0-4a66-b203-5e
7459225be3]   
F, [2019-09-16T14:07:02.113831 #233] FATAL -- : [b08fd659-a7f0-4a66-b203-5e
7459225be3] NoMethodError (undefined method `zero?' for nil:NilClass):
F, [2019-09-16T14:07:02.113927 #233] FATAL -- : [b08fd659-a7f0-4a66-b203-5e
7459225be3]   
F, [2019-09-16T14:07:02.114021 #233] FATAL -- : [b08fd659-a7f0-4a66-b203-5e
7459225be3] lib/not_at_origin_validator.rb:10:in `null_island?'
[b08fd659-a7f0-4a66-b203-5e7459225be3] lib/not_at_origin_validator.rb:6:in 
`validate'
[b08fd659-a7f0-4a66-b203-5e7459225be3] app/controllers/maps_controller.rb:4
7:in `create'

jywarren · 2019-09-16T14:34:57Z

Looks like paperclip now prefers aws-sdk-s3 gem instead of aws-sdk -- switching...

jywarren · 2019-09-16T14:39:18Z

thoughtbot/paperclip#2481

jywarren · 2019-09-16T15:57:41Z

Now, what's the other one...? where it fails if you're not logged in?

jywarren · 2019-09-16T16:24:38Z

Hmm. But still getting this when logged in:

F, [2019-09-16T16:22:44.877507 #293] FATAL -- : [eb43df33-0607-406a-999a-83
f0e986709f] NoMethodError (undefined method `id' for nil:NilClass): 
F, [2019-09-16T16:22:44.877574 #293] FATAL -- : [eb43df33-0607-406a-999a-83
f0e986709f]    
F, [2019-09-16T16:22:44.877652 #293] FATAL -- : [eb43df33-0607-406a-999a-83
f0e986709f] app/controllers/images_controller.rb:30:in `create'

jywarren · 2019-09-16T16:26:51Z

OK, so I think what's happening is that we need the params[:map_id] route param but we've moved the URL to:

http://mapknitter-stable.laboratoriopublico.org/maps/test3/edit

So, it's not getting it. Checking routes...

jywarren · 2019-09-16T16:32:07Z

Hmm, it looks like we're not getting proper map_id anywhere. I don't see /maps/____/edit in routes...

jywarren · 2019-09-16T16:36:35Z

Hm, just alerting @cesswairimu @sashadev-sky about this one. My thoughts are:

what routes are driving /maps/____/edit and how do we ensure map_id is properly plumbed?
does the maps create action forward you to /maps/_____/edit and is this correct?

For that last question, i see:

mapknitter/app/controllers/maps_controller.rb

Line 48 in c0b69f6

redirect_to "/maps/#{@map.slug}/edit"

jywarren · 2019-09-16T16:38:13Z

Update: creating a map when not logged in now works!

jywarren · 2019-09-16T16:40:07Z

So, i think we're not getting map_id on this line:

mapknitter/app/controllers/images_controller.rb

Line 29 in c0b69f6

map = Map.find_by(slug: params[:map_id])

jywarren · 2019-09-16T16:41:50Z

The request is to: http://mapknitter-stable.laboratoriopublico.org/maps/test4/warpables and got a 500 error.

jywarren · 2019-09-16T16:43:04Z

I think it uses this route:

mapknitter/config/routes.rb

Line 59 in c0b69f6

    
           post 'maps/:map_id/warpables', to: 'images#create' # deprecate this in favor of resourceful route below; this is just to override maps/:id

jywarren · 2019-09-16T16:49:53Z

D, [2019-09-16T16:41:12.742291 #826] DEBUG -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16]   ESC[1mESC[35m (34.6ms)ESC[0m  ESC[1mESC[31mROLLBA
CKESC[0m
I, [2019-09-16T16:41:12.743998 #826]  INFO -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16] Completed 500 Internal Server Error in 3581ms (Acti
veRecord: 91.3ms)
D, [2019-09-16T16:41:12.744318 #826] DEBUG -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16] undefined method `match' for nil:NilClass excluded 
from capture: DSN not set
F, [2019-09-16T16:41:12.745503 #826] FATAL -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16]   
F, [2019-09-16T16:41:12.745572 #826] FATAL -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16] NoMethodError (undefined method `match' for nil:Nil
Class):
F, [2019-09-16T16:41:12.745618 #826] FATAL -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16]   
F, [2019-09-16T16:41:12.745676 #826] FATAL -- : [f12ae9be-7345-4d89-8987-b542fa6e1f16] app/controllers/images_controller.rb:34:in `block i
n create'
[f12ae9be-7345-4d89-8987-b542fa6e1f16] app/controllers/images_controller.rb:33:in `create'

hmmmmm

jywarren · 2019-09-16T16:53:16Z

Could undefined method match' for nil:NilClassmean that, for whatever reason, we aren't getting something to matchformat` against?

jywarren · 2019-09-16T16:53:56Z

Ah, maybe format used to be part of the route? And now we aren't seeing it?

jywarren · 2019-09-16T16:57:40Z

Our generalized routes that included format have been removed as of Rails 5, so yeah, we may need to explicitly include that... checking the JS upload code...

Formerly:

  get ':controller/:action.:format'	
  get ':controller/:action/:id.:format'

jywarren · 2019-09-17T14:01:44Z

OK, locally, I'm able to upload images. So perhaps we're not looking at an issue with routes, but still with the aws-sdk-s3 gem configuration with Paperclip? I've added the keys in stable as environment variables, but not sure how the above errors relate to an AWS issue. Maybe I can configure my local setup to use AWS and reproduce the error from here...

jywarren · 2019-09-17T15:35:05Z

Great, reproduced locally:

[paperclip] Trying to link /tmp/19ef7b5ab18a8587c924d1af7403ac6f20190917-19818-qnkzgq.jpg to /tmp/19ef7b5ab18a8587c924d1af7403ac6f20190917-19818-1rnu9jh.jpg
Command :: file -b --mime '/tmp/19ef7b5ab18a8587c924d1af7403ac6f20190917-19818-1rnu9jh.jpg'
  Warpable Create (3.6ms)  INSERT INTO `warpables` (`image_content_type`, `image_file_name`, `image_file_size`, `created_at`, `updated_at`, `map_id`, `history`) VALUES ('image/png', 'IMG_20190914_175235.jpg', 412217, '2019-09-17 15:34:26', '2019-09-17 15:34:26', 3, 'None')                             
  ↳ app/controllers/images_controller.rb:34
  PaperTrail::Version Create (1.3ms)  INSERT INTO `versions` (`item_type`, `item_id`, `event`, `whodunnit`, `created_at`) VALUES ('Warpable', 8, 'create', '4', '2019-09-17 15:34:26')                                                                                                                        
  ↳ app/controllers/images_controller.rb:34
[paperclip] saving warpables/8/IMG_20190914_175235.jpg
   (6.2ms)  ROLLBACK
  ↳ app/controllers/images_controller.rb:34
Completed 500 Internal Server Error in 4883ms (ActiveRecord: 79.9ms)


undefined method `match' for nil:NilClass excluded from capture: DSN not set
  
NoMethodError (undefined method `match' for nil:NilClass):
  
app/controllers/images_controller.rb:34:in `block in create'
app/controllers/images_controller.rb:33:in `create'

jywarren · 2019-09-17T18:22:29Z

OK, got a better trace of the error:

NoMethodError in ImagesController#create

undefined method `match' for nil:NilClass
Extracted source (around line #82):

#80       def partition_matching_region(region)
#81         @rules['partitions'].find do |p|
*82           region.match(p["regionRegex"]) ||
#83           p['services'].values.find { |svc| svc['endpoints'].key?(region) if svc.key? 'endpoints' }
#84         end
#85       end

Extracted source (around line #81):

#79 
#80       def partition_matching_region(region)
*81         @rules['partitions'].find do |p|
#82           region.match(p["regionRegex"]) ||
#83           p['services'].values.find { |svc| svc['endpoints'].key?(region) if svc.key? 'endpoints' }
#84         end

jywarren · 2019-09-17T18:23:52Z

Same error as here! ankane/pghero#69 --

Try setting ENV["AWS_REGION"]

wow, trying now!

jywarren · 2019-09-17T18:30:11Z

mapknitter/config/amazon_s3.yml

Line 12 in 6642c6c

s3_region: <%= ENV['AWS_REGION'] || 'us-east-1' %>

6642c6c

jywarren · 2019-09-17T18:57:03Z

Yes!!!

[paperclip] saving warpables/13/IMG_20190914_175235.jpg
[paperclip] saving warpables/13/IMG_20190914_175235_medium.jpg
[paperclip] saving warpables/13/IMG_20190914_175235_small.jpg
[paperclip] saving warpables/13/IMG_20190914_175235_thumb.jpg
[paperclip] copying warpables/13/IMG_20190914_175235.jpg to local file /tmp/19ef7b5ab18a8587c924d1af7403ac6f20190917-4640-1tprhbj.jpg
Command :: identify -format '%wx%h,%[exif:orientation]' '/tmp/19ef7b5ab18a8587c924d1af7403ac6f20190917-4640-1tprhbj.jpg[0]' 2>/dev/null
   (5.8ms)  ROLLBACK
  ↳ app/controllers/images_controller.rb:33
Completed 500 Internal Server Error in 57603ms (ActiveRecord: 101.8ms)


Paperclip::Errors::NotIdentifiedByImageMagickError excluded from capture: DSN not set
  
Paperclip::Errors::NotIdentifiedByImageMagickError (Paperclip::Errors::NotIdentifiedByImageMagickError):
  
app/models/warpable.rb:57:in `save_dimensions'
app/controllers/images_controller.rb:33:in `create'

jywarren · 2019-09-17T22:17:41Z

OK, now we are seeing something different on stable (after adding the REGION):

I, [2019-09-17T22:16:32.615173 #232]  INFO -- : [7f81fc93-0253-40e0-bf58-3b4d57b5e7ec] [paperclip] saving warpables/3/w130-example.jpg
D, [2019-09-17T22:16:35.673031 #232] DEBUG -- : [7f81fc93-0253-40e0-bf58-3b4d57b5e7ec]   ESC[1mESC[35m (47.1ms)ESC[0m  ESC[1mESC[31mROLLBACKESC[0m
I, [2019-09-17T22:16:35.674833 #232]  INFO -- : [7f81fc93-0253-40e0-bf58-3b4d57b5e7ec] Completed 500 Internal Server Error in 3703ms (ActiveRecord: 111.2ms)
D, [2019-09-17T22:16:35.675517 #232] DEBUG -- : [7f81fc93-0253-40e0-bf58-3b4d57b5e7ec] missing credentials, provide credentials with one of the following options:
  - :access_key_id and :secret_access_key
  - :credentials
  - :credentials_provider excluded from capture: DSN not set
F, [2019-09-17T22:16:35.678102 #232] FATAL -- : [7f81fc93-0253-40e0-bf58-3b4d57b5e7ec]   
F, [2019-09-17T22:16:35.678193 #232] FATAL -- : [7f81fc93-0253-40e0-bf58-3b4d57b5e7ec] Aws::Sigv4::Errors::MissingCredentialsError (missing credentials, provide credentials with one of the following options:
  - :access_key_id and :secret_access_key
  - :credentials
  - :credentials_provider):

jywarren · 2019-09-17T22:29:57Z

To reproduce, try uploading an image to: http://mapknitter-stable.laboratoriopublico.org/maps/test3/edit

icarito · 2019-09-18T04:07:28Z

I believe this should be solved with referencing the variables as in #1034 to be available on the container.

jywarren · 2019-09-18T15:41:07Z

Thanks! I think there's another follow-on bug, so just going to document here and re-open:

D, [2019-09-18T15:38:09.367663 #232] DEBUG -- : [e3d39e56-d90f-4454-abd2-36368509b1ec]   ESC[1mESC[35m (0.5ms)ESC[0m  ESC[1mESC[35mBEGINESC[0m
I, [2019-09-18T15:38:09.368912 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] [paperclip] Trying to link /tmp/e8253a654a2c768887817a08c1fa2ac6
20190918-232-wyyj1w.jpeg to /tmp/e8253a654a2c768887817a08c1fa2ac620190918-232-a8tqyg.jpeg
I, [2019-09-18T15:38:09.369867 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] Command :: file -b --mime '/tmp/e8253a654a2c768887817a08c1fa2ac6
20190918-232-a8tqyg.jpeg'
D, [2019-09-18T15:38:09.391237 #232] DEBUG -- : [e3d39e56-d90f-4454-abd2-36368509b1ec]   ESC[1mESC[36mWarpable Create (1.0ms)ESC[0m  ESC[1mESC[32mINSER
T INTO `warpables` (`image_content_type`, `image_file_name`, `image_file_size`, `created_at`, `updated_at`, `map_id`, `history`) VALUES ('image/jpeg', 
'DJI_1202_medium.jpeg', 222065, '2019-09-18 15:38:09', '2019-09-18 15:38:09', 2, 'None')ESC[0m
D, [2019-09-18T15:38:09.411930 #232] DEBUG -- : [e3d39e56-d90f-4454-abd2-36368509b1ec]   ESC[1mESC[36mPaperTrail::Version Create (0.6ms)ESC[0m  ESC[1m
ESC[32mINSERT INTO `versions` (`item_type`, `item_id`, `event`, `whodunnit`, `created_at`) VALUES ('Warpable', 1, 'create', '1', '2019-09-18 15:38:09')
ESC[0m
I, [2019-09-18T15:38:09.412985 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] [paperclip] saving warpables/1/DJI_1202_medium.jpeg
I, [2019-09-18T15:38:09.680752 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] [paperclip] saving warpables/1/DJI_1202_medium_medium.jpeg
I, [2019-09-18T15:38:09.813509 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] [paperclip] saving warpables/1/DJI_1202_medium_small.jpeg
I, [2019-09-18T15:38:09.878030 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] [paperclip] saving warpables/1/DJI_1202_medium_thumb.jpeg
D, [2019-09-18T15:38:10.025126 #232] DEBUG -- : [e3d39e56-d90f-4454-abd2-36368509b1ec]   ESC[1mESC[35m (77.7ms)ESC[0m  ESC[1mESC[31mROLLBACKESC[0m
I, [2019-09-18T15:38:10.027057 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] Completed 500 Internal Server Error in 2425ms (ActiveRecord: 161
.2ms)

...

I, [2019-09-18T15:38:10.093523 #232]  INFO -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] Sending event a6f031a7dcf74f6da79a0ea65762af7d to Sentry
F, [2019-09-18T15:38:10.319091 #232] FATAL -- : [e3d39e56-d90f-4454-abd2-36368509b1ec]
F, [2019-09-18T15:38:10.319223 #232] FATAL -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] Paperclip::AdapterRegistry::NoHandlerError (No handler found for
 "https://s3.amazonaws.com/grassrootsmapping/warpables/1/DJI_1202_medium.jpeg"):
F, [2019-09-18T15:38:10.319312 #232] FATAL -- : [e3d39e56-d90f-4454-abd2-36368509b1ec]
F, [2019-09-18T15:38:10.319411 #232] FATAL -- : [e3d39e56-d90f-4454-abd2-36368509b1ec] app/models/warpable.rb:55:in `save_dimensions'
[e3d39e56-d90f-4454-abd2-36368509b1ec] app/controllers/images_controller.rb:34:in `block in create'
[e3d39e56-d90f-4454-abd2-36368509b1ec] app/controllers/images_controller.rb:33:in `create'
I, [2019-09-18T15:38:10.516723 #232]  INFO -- : [30a1931a-6319-40a3-b829-b4f2ff2503c1] Started GET "/cable" for 207.255.187.134 at 2019-09-18 15:38:10
+0000
I, [2019-09-18T15:38:10.518562 #232]  INFO -- : [30a1931a-6319-40a3-b829-b4f2ff2503c1] Started GET "/cable/"[non-WebSocket] for 207.255.187.134 at 2019
-09-18 15:38:10 +0000
E, [2019-09-18T15:38:10.518690 #232] ERROR -- : [30a1931a-6319-40a3-b829-b4f2ff2503c1] Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNEC
warren@tycho:/srv/mapknitter_stable/mapknitter$ clear

Looks like the key is here:

Paperclip::AdapterRegistry::NoHandlerError (No handler found for "https://s3.amazonaws.com/grassrootsmapping/warpables/1/DJI_1202_medium.jpeg"):

jywarren · 2019-09-18T15:55:55Z

Locally i get:


Paperclip::Errors::NotIdentifiedByImageMagickError excluded from capture: DSN not set

Paperclip::Errors::NotIdentifiedByImageMagickError (Paperclip::Errors::NotIdentifiedByImageMagickError):

app/models/warpable.rb:57:in `save_dimensions'
app/controllers/images_controller.rb:33:in `create'

jywarren · 2019-09-18T16:14:33Z

OK, great. I just got locally to the same error as in stable:


No handler found for "https://s3.amazonaws.com/grassrootsmapping/warpables/15/w130-example.jpg" excluded from capture: DSN not set
  
Paperclip::AdapterRegistry::NoHandlerError (No handler found for "https://s3.amazonaws.com/grassrootsmapping/warpables/15/w130-example.jpg"):
  
app/models/warpable.rb:55:in `save_dimensions'

jywarren · 2019-09-19T13:35:33Z

Maybe this??? thoughtbot/paperclip#2198

jywarren · 2019-09-19T14:23:19Z

I think i got this!

jywarren added the bug label Sep 16, 2019

jywarren mentioned this issue Sep 16, 2019

s3 fix #1032

Merged

jywarren closed this as completed in #1032 Sep 16, 2019

jywarren reopened this Sep 16, 2019

jywarren added the priority label Sep 16, 2019

icarito mentioned this issue Sep 18, 2019

Add env vars #1034

Merged

5 tasks

jywarren closed this as completed in #1034 Sep 18, 2019

jywarren reopened this Sep 18, 2019

jywarren mentioned this issue Sep 19, 2019

s3 fix #1040

Merged

jywarren closed this as completed in #1040 Sep 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bugs on stable in map creation #1031

Bugs on stable in map creation #1031

jywarren commented Sep 16, 2019 •

edited

Loading

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

icarito commented Sep 18, 2019

jywarren commented Sep 18, 2019 •

edited

Loading

jywarren commented Sep 18, 2019

jywarren commented Sep 18, 2019

jywarren commented Sep 19, 2019

jywarren commented Sep 19, 2019

Bugs on stable in map creation #1031

Bugs on stable in map creation #1031

Comments

jywarren commented Sep 16, 2019 • edited Loading

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 16, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

jywarren commented Sep 17, 2019

icarito commented Sep 18, 2019

jywarren commented Sep 18, 2019 • edited Loading

jywarren commented Sep 18, 2019

jywarren commented Sep 18, 2019

jywarren commented Sep 19, 2019

jywarren commented Sep 19, 2019

jywarren commented Sep 16, 2019 •

edited

Loading

jywarren commented Sep 18, 2019 •

edited

Loading