Updated API #6

xSAVIKx · 2016-02-22T13:09:54Z

I've update java API, so now it's up-to-date with rubi API.
Added spock unit tests and one functional test stub.
moved project from ANT to Gradle.
introduced separate tasks to generate javaDocs and OneJar (with all needed dependencies).
moved project from custom XML-parser to general, easy-to-update JAXB solution.
added support for future error-handling and provided custom TwinglyException wrapper for any exception happened.

added initial gradle structure and renamed package.

added test dependencies to gradle build added settings.gradle file

use JAXB instead of custom SAX parser remove most parameters from Query fields and move them to methods as arguments. add Language enum to provide all possible supported languages removed not used anymore files updated gradle build

added unit tests written with Spock to provide code coverage added javadoc to each class added gradle task to generate javadoc generated javadoc

added exception handling support and changed all runtime exceptions to custom TwinglyException. changed XML fetching to use custom User agent

remove javadoc folder from repo, cause it's easy to generate update build.gradle with prettier task names and doc location property update readme with up-to-date examples

walro · 2016-02-23T08:27:08Z

src/test/java/com/twingly/search/QueryFunctionalTest.java

@@ -0,0 +1,37 @@
+package com.twingly.search;


How are these tests run, it seems like they need to be manually invoked now?

walro · 2016-02-23T08:32:12Z

I like the enum for the languages in Language.java, but we add support for new languages from time to time. The changes proposed here forces a release of a new version, with the languages added to the enum. I think that we need to keep it a bit more open, perhaps the enum could be used as a helper, but you are still able to use arbitrary language codes.

walro · 2016-02-23T08:34:17Z

I also think it might make sense the separate the client from the query as we have done in the Ruby case (split into two classes, Client and Query). Keep the network related code in the Client and the logics adhering to creating/formatting the query in the Query class. WDYT of that?

now all processes related to making request is done inside client, so that query object is now responsible for Query creation and delegating other request stuff to client. changed Language enum, so now it won't through exception, but will return null, if no language for ISO code was found. changed functionality of Query creation, so that it could be created with custom language - now your should pass String object instead of Language enum instance to build query.

added access to execute gradlew

added ability to use create query object with properly set environment variable with Twingly API key. added betamax dependencies to gradle build added client javadoc

walro · 2016-02-24T08:52:04Z

src/main/java/com/twingly/search/Query.java

+ * The type Query.
+ */
+public class Query {
+    private static final String BASE_URL = "http://api.twingly.com";


I noticed it was changed from https to http in an earlier commit, what didn't work with https?

Betamax won't work with https right now, or maybe some changes to client should be done.
Actually, I've found that Betamax should work well on https with apache httpclient, but it's over 700kb without its own dependencies and I think you don't need it.)
Maybe with betamax update the situation will change.

Ah, I see. I think we can live with large dependencies if that is what it takes to get https going. If it's just betamax, would it be possible to just pull those dependencies in for just the tests?

Unfortunately - no.
If we want to use betamax with https right now (actually, they are going to publish new beta in the beginning of march) - we need to use httpclient also to retrieve data from API.
(It's how I see it now.)

I see, Betamax will use what the Twingly Search API client uses.

Exactly what are the problem with HTTPS and Betamax? Is it something with our SSL setup on the server-side?

Just looking at "IMPLEMENTATIONS" at http://betamax.software/ it sounds like it should work with both implementations they present.

@xSAVIKx but you only have the problem when using betamax? otherwise the client works fine with https, correct?

@dentarg Yeap, it's only related to Betamax usage.
Without betamax testing everything is working perfectly with URLConnection or Apache HTTP clients

To get things progressing, I think we'll have to live without Betamax for now. We need https support. Ruby tests using VCR may be implemented/ported using mocks or appropriate fixtures instead for the time being.

Also, I think the constants BASE_URL and SEARCH_PATH might be more suited in the Client class. Mostly due to that how we have split in other API clients, but I think it makes sense :)

I'll also play today with mock-server, maybe it'll be more suitable for such tests.

Great! 👍

dentarg · 2016-02-24T13:16:37Z

I played around with the gradle wrapper (I understand why this is the future over maven 😄) and when running the tests (gradlew check and gradlew test) I noticed there wasn't any output from the tests. Example of what output I refer to can be seen at https://travis-ci.org/twingly/twingly-search-api-ruby/jobs/109909967. That's one of the reason we like rspec (for Ruby) and behaviour testing. Is there any way to achieve that for the tests here? Perhaps the tests aren't running yet?

add tests logging

xSAVIKx · 2016-02-24T14:36:29Z

I'll play with gradle tests output.

adjusted test logging level

walro · 2016-02-29T08:30:58Z

build.gradle

+test {
+    testLogging.showStandardStreams = true
+    testLogging {
+        events "PASSED", "STARTED", "FAILED", "SKIPPED"


Nice with output but I think that STARTED is not needed

@walro , IMO it has clearer view, when you see that it was started and, after that, the result, but if you want - I'll remove started event.

Might be that I am more used to just seeing one line per test. We can always add STARTED back later on, if we change our minds.

added +x permission to gradlew and gradlew.bat and removed chmod command from "before_install" section in travis config

created 2 implementations for Client interface updated spock tests due to new approach changed twingly protocol from http to https removed betamax from dependencies and repositories

update build.gradle for future easier publishing update readme to be more correct update clients javadoc

added xml results from ruby repo and tested them against new Java API

twingly-mob · 2016-03-02T09:32:23Z

@xSAVIKx why is there support for two different HttpClients? ApacheHttpClient and UrlConnectionClient. It seems like the latter is the default client. It's less maintenance for us to just keep one around, that would also remove the need for the AbstractClient and possibly even the interface.

… refactorings ensure that every file in repo ends with new line remove obsolete LICENSE.txt file update copyright date in readme update readme with gradlew usage instead of gradle in examples

dentarg · 2016-03-02T09:38:05Z

@xSAVIKx okay, thanks

xSAVIKx · 2016-03-02T09:38:17Z

@twingly-mob
They were created for test purposes, while I was testing betamax usage.
As for the clients: UrlConnectionClient is more simple, but have less possibilities to be configured.
And for the interface - it's always better to use interface than implementation for clearer usage and better test-possibilities.
If you want me to remove one of clients - OK, I'll do it.

twingly-mob · 2016-03-02T09:41:04Z

The relationship between Client and Query objects seems a bit off. We think it would make more sense that the end-user explicitly instantiates the Client object and not the Query object. We do this in our other clients, for instance: https://github.com/twingly/twingly-search-api-python#usage and https://github.com/twingly/twingly-search-api-php#usage.

Also, the Query class holds the BASE_URL and SEARCH_PATH constants, those would make more sense in the Client or the Constants class.

twingly-mob · 2016-03-02T09:42:12Z

And for the interface - it's always better to use interface than implementation for clearer usage and better test-possibilities.
If you want me to remove one of clients - OK, I'll do it

Yes, please remove one of them. It doesn't really matter that much which one - keep the one you like the most. :)

xSAVIKx · 2016-03-02T09:48:48Z

@twingly-mob
It's not good Java style to get Query from Client, while Query won't be used inside Client...

xSAVIKx · 2016-03-02T09:53:31Z

I'd suggest to make some QueryBuilder that will generate valid Query object and than use it with Client, like:

Client client = new Client();
Query query = QueryBuilder.create(apiKey).setLanguage(language).setStartTime(startTime).setEndTime(endTime).setSearchPattern(searchPattern).build();
Result result = client.makeRequest(query);

Or smth similar to that. It's kind of correct style.

xSAVIKx · 2016-03-02T09:56:27Z

There's no need, actually, to handle Client instance inside Query object, moreover, IMO, it's logically wrong to do so.

twingly-mob · 2016-03-02T09:57:01Z

I'd suggest to make some QueryBuilder that will generate valid Query object and than use it with Client, like:

This was basically exactly what we just were discussing here. :) Our other clients should probably look a bit more like what you just proposed.

twingly-mob · 2016-03-02T10:01:21Z

There's no need, actually, to handle Client instance inside Query object, moreover, IMO, it's logically wrong to do so.

Yeah, we added the execute method on the query for convenience, but as you said, it introduced two-way-coupling, the client needs to know about queries and vice-versa.

xSAVIKx · 2016-03-02T10:01:22Z

I agree, but, you know, it smells like CR, actually...
I'll change it, but, you know. it should be planned before implementation started.

twingly-mob · 2016-03-02T10:02:58Z

I agree, but, you know, it smells like CR, actually...
I'll change it, but, you know. it should be planned before implementation started.

Agreed, we can discuss this on Upwork.

xSAVIKx · 2016-03-02T10:03:37Z

OK, lets do it.

removed ApacheHttpClient and merged AbstractClient functionality with UrlConnectionClient. merged tests from AbstractClientUnmarshallingSpockTest to urlConnectionClientSpockTest

Extend exception with more speialized ones as: TwinglyException -> TwinglySearchException (just renamed) TwinglySearchServerException - super class for all server exceptions TwinglySearchServerAPIKeyDoesNotExistException - should be thrown when no API key was found TwinglySearchServerAPIKeyUnauthorizedException - should be thrown when API key is not authorized for any action TwinglySearchServerServiceUnavailableException - should be thrown when service is not available Refactor models, so that all models are now stored under domain package. Extended OperationResult with OperationResultType enum for easier usage.

…on for non-valid content types

updated example to fetch all possible results up to last publish date added helper method "areAllResultsReturned" to Result class added NPE check for languages in Query class

Updated API

xSAVIKx added 6 commits February 18, 2016 23:59

initial commit. rename package due to #3

ab9fb56

added initial gradle structure and renamed package.

update gradle build information

bfc296e

added test dependencies to gradle build added settings.gradle file

code refactored hardly

0fd6654

use JAXB instead of custom SAX parser remove most parameters from Query fields and move them to methods as arguments. add Language enum to provide all possible supported languages removed not used anymore files updated gradle build

added unit tests and javadoc

8d5065e

added unit tests written with Spock to provide code coverage added javadoc to each class added gradle task to generate javadoc generated javadoc

added exception handling support

b0a326a

added exception handling support and changed all runtime exceptions to custom TwinglyException. changed XML fetching to use custom User agent

remove javadoc from repo and update readme

021c857

remove javadoc folder from repo, cause it's easy to generate update build.gradle with prettier task names and doc location property update readme with up-to-date examples

walro reviewed Feb 23, 2016
View reviewed changes

xSAVIKx and others added 6 commits February 23, 2016 16:23

check in travis CI config file

8a08295

Update .travis.yml

81acc40

added access to execute gradlew

Update .travis.yml

f69ba59

add build status to readme

6f077a8

added support for env API key. added betamax dependencies

a07a499

added ability to use create query object with properly set environment variable with Twingly API key. added betamax dependencies to gradle build added client javadoc

walro reviewed Feb 24, 2016
View reviewed changes

Update build.gradle

37fcf95

add tests logging

added groovy support for spock tests

b5fbfa4

adjusted test logging level

walro reviewed Feb 29, 2016
View reviewed changes

xSAVIKx and others added 7 commits February 29, 2016 19:54

fix gradlew execution permissions

56540b9

added +x permission to gradlew and gradlew.bat and removed chmod command from "before_install" section in travis config

fix gradlew execution permissions

a476f32

added +x permission to gradlew and gradlew.bat and removed chmod command from "before_install" section in travis config

Merge remote-tracking branch 'origin/master'

9514010

Updated gradlew and gradlew.bat permissions from ubuntu

d8ec63e

refactored clients and removed betamax from dependencies

97bacfe

created 2 implementations for Client interface updated spock tests due to new approach changed twingly protocol from http to https removed betamax from dependencies and repositories

updated build for future publising, updated javadoc and readme

9b61fee

update build.gradle for future easier publishing update readme to be more correct update clients javadoc

added fixture tests

edd3b90

added xml results from ruby repo and tested them against new Java API

ensure file ends with new line, remove license obsolete file and misc…

2eb1db8

… refactorings ensure that every file in repo ends with new line remove obsolete LICENSE.txt file update copyright date in readme update readme with gradlew usage instead of gradle in examples

twingly-mob mentioned this pull request Mar 2, 2016

Remove one of the two http clients xSAVIKx/twingly-search-api-java#1

Closed

xSAVIKx added 11 commits March 2, 2016 14:15

#1 - removed ApacheHttpClient as obsolete

f055f46

removed ApacheHttpClient and merged AbstractClient functionality with UrlConnectionClient. merged tests from AbstractClientUnmarshallingSpockTest to urlConnectionClientSpockTest

#3 update readme with new exception types

6cce8c5

#4 #7 - added lists initialization in Post and Result. Added filtrati…

4ad5066

…on for non-valid content types

#5 extended test to test all properties of minimal valid result

2c8a771

#6 Empty searchpattern should throw exception

2029b44

#5 extended test to test all properties of minimal valid result

31479d4

#8 updated system property for twingly search key and readme

23091f2

added examples

82ed813

update build.gradle for example to fix build

a4d2445

updated SearchPostsStream example

aefe054

updated example to fetch all possible results up to last publish date added helper method "areAllResultsReturned" to Result class added NPE check for languages in Query class

twingly-mob added a commit that referenced this pull request Mar 4, 2016

Merge pull request #6 from xSAVIKx/master

bc6030c

Updated API

twingly-mob merged commit bc6030c into twingly:master Mar 4, 2016

This was referenced Mar 4, 2016

Test suite #5

Closed

Rename to twingly-search-api-java? #3

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated API #6

Updated API #6

xSAVIKx commented Feb 22, 2016

walro Feb 23, 2016

walro commented Feb 23, 2016

walro commented Feb 23, 2016

walro Feb 24, 2016

xSAVIKx Feb 24, 2016

walro Feb 24, 2016

xSAVIKx Feb 24, 2016

dentarg Feb 24, 2016

dentarg Feb 26, 2016

xSAVIKx Feb 27, 2016

walro Feb 29, 2016

xSAVIKx Feb 29, 2016

walro Feb 29, 2016

dentarg commented Feb 24, 2016

xSAVIKx commented Feb 24, 2016

walro Feb 29, 2016

xSAVIKx Feb 29, 2016

walro Feb 29, 2016

twingly-mob commented Mar 2, 2016

dentarg commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

Updated API #6

Updated API #6

Conversation

xSAVIKx commented Feb 22, 2016

Choose a reason for hiding this comment

walro commented Feb 23, 2016

walro commented Feb 23, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dentarg commented Feb 24, 2016

xSAVIKx commented Feb 24, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

twingly-mob commented Mar 2, 2016

dentarg commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016

twingly-mob commented Mar 2, 2016

xSAVIKx commented Mar 2, 2016