Upsource 2017.3 with Gitlab integration issues

Hi, 

we upgraded our upsource installation to 2017.3 and tried to integrate with our Gitlab server. But somehow it is not working correctly. 

Here are the steps we did so far. 

1. We created a upsource user in Gitlab and generate a personal access token for that user.

2. In upsource (in project configuration .../administration/settings?tab=general) we checked the two checkboxes "Import merge requests as branches" and "Synchronize comments and merge requests" and entered the token we created for the gitlab user into the field that occurred with the second check box. 

3. We logged out of gitlab with the upsource user and logged in again with a normal gitlab user. 

4. In upsource we then selected a branch (.../branch/<branch_name>) and chose the option "Create merge request..." and entered all necessary fields. After pressing "Create" upsource shows following error "Remote GitLab operation failed: I/O error occurred"

 

Here is an extract from the logs

----

[2017-12-15 15:23:06,820] [Upsource Frontend Error] [2017-12-15 15:23:06,819] WARN gitlab-sync-3 .integration.gitlab.GitlabHttp - Access token doesn’t look like neither OAuth nor private token. Using OAuth
[2017-12-15 15:23:06,851] [Upsource Frontend Error] [2017-12-15 15:23:06,842] WARN gitlab-sync-3 ion.gitlab.GitlabImportService - Import failed: YS
[2017-12-15 15:23:06,851] [Upsource Frontend Error] com.jetbrains.upsource.UpsourceException: Import failed: YS
[2017-12-15 15:23:06,851] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabImporter.importAll(GitlabImporter.java:92)
[2017-12-15 15:23:06,851] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabImportService.a(GitlabImportService.java:147)
[2017-12-15 15:23:06,851] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.core.db.ProjectHolder.with2(ProjectHolder.kt:95)
[2017-12-15 15:23:06,851] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabImportService.a(GitlabImportService.java:145)
[2017-12-15 15:23:06,851] [Upsource Frontend Error] at java.util.HashMap$KeySet.forEach(HashMap.java:933)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabImportService.a(GitlabImportService.java:112)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at com.jetbrains.upsource.util.NamedDaemonThreadFactory.a(NamedDaemonThreadFactory.java:34)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at java.lang.Thread.run(Thread.java:748)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] Caused by: java.lang.Error: java.net.ConnectException: Connection refused (Connection refused)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp$1.b(GitlabHttp.java:167)
[2017-12-15 15:23:06,852] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp$1.hasNext(GitlabHttp.java:125)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp.getAll(GitlabHttp.java:105)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabApiClient.getMergeRequests(GitlabApiClient.java:36)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabImporter.c(GitlabImporter.java:102)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabImporter.importAll(GitlabImporter.java:90)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] ... 13 more
[2017-12-15 15:23:06,853] [Upsource Frontend Error] Caused by: java.net.ConnectException: Connection refused (Connection refused)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at java.net.PlainSocketImpl.socketConnect(Native Method)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[2017-12-15 15:23:06,853] [Upsource Frontend Error] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at java.net.Socket.connect(Socket.java:589)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
[2017-12-15 15:23:06,854] [Upsource Frontend Error] at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1564)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:347)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp.a(GitlabHttp.java:229)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp.access$100(GitlabHttp.java:26)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp$1.b(GitlabHttp.java:163)
[2017-12-15 15:23:06,855] [Upsource Frontend Error] ... 18 more

----

 

 

We're not really sure whether the setup like we did it  is correct. Is there a more detailed documentation? 

 

 

 

Regards, 

Flo

 

 

11 comments
Comment actions Permalink

Hi Florian,

We are missing documentation page for GitLab synchronization. Once it's ready I'll let you know.

The question I have is the following - have you configured GitLab auth module in Hub?

0
Comment actions Permalink

Yes we configured the auth module for GitLab. 

We created a upsource user in GitLab, went to the settings of that user and registered an application. The credentials (client id, client secret) for this application we entered in the auth module configuration. Beside this we adapted the urls to point to our GitLab server.  

The app on GitLab has following scopes/persmissions

  • api
  • read_user
  • sudo
  • openid
0
Comment actions Permalink

Hey Florian,

Sorry for the delay. The doc page is now ready, steps are quite simple and it looks like you have already completed all of them:

https://www.jetbrains.com/help/upsource/2017.3/synchronizing-a-gitlab-project.html

Logs you posted before mentioning some errors about auth token, that said, could you try "test connection" on the admin page one more time and check the result?

0
Comment actions Permalink

Thanks. 

Test connection says "VCS connection successful". 

Maybe also interesting information, we use GitLab with LDAP authentification. 

But it still does not work.  

 

 

0
Comment actions Permalink

Ok, one more question, how have you created the token? Manually on the GitLab side or by clicking Acquire synchronization token?

0
Comment actions Permalink

Actually I tried both. 

I also tried different users with different permissions in gitlab for the a repository. 

 

User 1:

- Authentication mode in GitLab: Login/Password 

- Developer role in the Project/Repository (in GitLab)

When I use this user to acquire synchronization token, I get following error. 

 

User 2: 

- Authentication mode in GitLab: LDAP 

- Master role in the the Project/Repository (in GitLab)

When I use this user to acquire synchronization token, I get a token and can save the configuration page. 

I then go to the project page in up source and press the gitlab button on the top right and get authenticated. 

I go to a branch in up source and press the "Create merge request" button. 

I enter a name for the merge request in the pop up and press ok. 

=> Following error pops up "Remote GitLab operation failed: I/O error occurred"

In the log I can see following warning with stack trace. 

---

[2018-01-19 15:02:37,253] WARN RPC task executor-5 egration.gitlab.GitlabExporter - Connection refused (Connection refused)
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:673)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:264)
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1156)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1334)
at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1309)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:259)
at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp.request(GitlabHttp.java:86)
at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabHttp.post(GitlabHttp.java:67)
at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabApiClient.createMergeRequest(GitlabApiClient.java:48)
at com.jetbrains.upsource.backend.server.integration.gitlab.GitlabExporter.createMergeRequest(GitlabExporter.java:174)
at com.jetbrains.upsource.frontend.server.handlers.rpc.impl.commands.CreatePullRequestCommandImpl.execute(CreatePullRequestCommandImpl.java:101)
at com.jetbrains.upsource.frontend.server.handlers.rpc.impl.commands.CreatePullRequestCommandImpl.execute(CreatePullRequestCommandImpl.java:54)
at com.jetbrains.upsource.frontend.server.taskEngine.TaskRunnable.b(TaskRunnable.java:57)
at com.jetbrains.upsource.stats.StatsManager.decorateCompletableFuture(StatsManager.java:218)
at com.jetbrains.upsource.stats.StatsManager.decorateCompletableFuture(StatsManager.java:171)
at com.jetbrains.upsource.frontend.server.taskEngine.TaskRunnable.run(TaskRunnable.java:56)
at com.jetbrains.upsource.stats.StatsManager.a(StatsManager.java:156)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at com.jetbrains.upsource.util.NamedDaemonThreadFactory.a(NamedDaemonThreadFactory.java:34)
at java.lang.Thread.run(Thread.java:748)

---

 

I wonder why the call goes through a class called SSLSocketImpl? Our services are all running under http, not https.

 

 

 

 

 

0
Comment actions Permalink

I changed role of the first user from Developer to Master, but I still cannot get the synchronization token.

 

Another thing that is also not 100% clear to me is, what is the actual idea of the git lab integration? What can be done and what cannot be done, given that the integration ist working. 

Instead of clicking on "Create a merge request" button I chose the option "Create a branch review" from the drop down menu. Now the "Create a branch review" button is gone. 

0
Comment actions Permalink

Florian, I was having the same issue and the problem is right there with https. So to solve the problem I enabled https for my gitlab server and all started working. Hope this helps anybody.

0
Comment actions Permalink

Thanks, we will give it a try. 

0
Comment actions Permalink

And for those of us who aren't at liberty to enable https for our gitlab server... tough luck?

0

Please sign in to leave a comment.