From 463d7da0f27f45ec8941a4546caabe12adfaa35a Mon Sep 17 00:00:00 2001 From: Seth Call Date: Tue, 17 Dec 2013 22:34:31 +0000 Subject: [PATCH 01/20] * make tester page look a little more user-friendly --- web/app/views/ping/ping.html.erb | 70 ++++++++++++++++++++++++++------ 1 file changed, 57 insertions(+), 13 deletions(-) diff --git a/web/app/views/ping/ping.html.erb b/web/app/views/ping/ping.html.erb index 4f6bf6bb7..728b9be04 100644 --- a/web/app/views/ping/ping.html.erb +++ b/web/app/views/ping/ping.html.erb @@ -1,25 +1,69 @@ Test Internet Latency + -

Test Internet Latency

-

Select the link corresponding to your internet service provider. - This will launch an applet to test the performance of your connection.

-

My ISP is AT&T

-

Click <%= link_to 'here', '/ping/pingat.jnlp' %>.

+

Internet Speed Test

+
-

My ISP is Comcast

-

Click <%= link_to 'here', '/ping/pingcc.jnlp' %>.

+

Welcome, and thank you for helping us by running this quick latency test application. The app may just run, or you might need to install or update the version of Java on your computer. It will take just one minute if you have Java already installed and up-to-date, or less than five minutes if you have to install or update Java on your computer.

-

My ISP is Time Warner

-

Click <%= link_to 'here', '/ping/pingtw.jnlp' %>.

+

Following are step-by-step directions to run this test:

+
    +
  1. Please run this test app from your home, not from a business or coffee shop or anywhere else - only from your home, as we need the data collected from home environments.
    2. -

    My ISP is Verizon

    -

    Click <%= link_to 'here', '/ping/pingvz.jnlp' %>.

    +
  2. Please run this test app on a Windows computer. It’s too hard to get it to run on a Mac, even though it’s theoretically possible.
    3. -

    My ISP is none of the above.

    -

    Click <%= link_to 'here', '/ping/pingno.jnlp' %>.

    +
  3. Please connect your Windows computer to your home router with an Ethernet cable rather than connecting wirelessly via WiFi. This is important to the accuracy of the data being collected, thank you!
    4. + +
  4. To start the test, please click the Run Test button on the right side of this page next to the ISP that provides Internet service to your home.
    5. + +
  5. When you click the Run Test button, a file will start to download in your browser. It may display a message like “This type of file can harm your computer. Do you want to keep it anyway?”. Please click the button that lets your browser go ahead and download and save the file. It’s just a little test app we wrote ourselves, so we know it’s safe, but we did not sign the app with a certificate, so you may get this warning message.
    6. + +
  6. When the file is downloaded, please click on the file to open and run it. If your version of Java is up-to-date, the app will run as described in step 7 below. If you get a message that Java is not up-to-date on your computer, please follow step 6 below to update Java on your computer.
    7. + +
  7. Click the prompt button to update Java. You are taken to the Java website. Click the red Free Java Download button. Then click the Agree & Start Free Download button. When the file is downloaded, click on the file to open/run it, and then follow the on-screen instructions to install the Java update. (Note: Watch out during the installation for the McAfee option, and uncheck that one to avoid getting McAfee installed on your computer.) After the Java update has installed, go back to the JamKazam test app webpage, and click on the Run Test button again next to the ISP that provides Internet service to your home.
    8. + +
  8. When you run the test app, a window will open and you’ll be prompted “Do you want this app to run?”. Please answer yes to let the app run. Then you’ll see a small window open, and you’ll see the test app running. This will take less than a minute. When it’s finished, it displays “Results posted, thank you for your time!”. You can close the window, and you are all done.
    9. +
+ +

Thanks again very much for helping us collect this Internet latency data!

+ +

Regards,
+ The JamKazam Team

+
+
+

Select the link corresponding to your internet service provider. + This will launch an applet to test the performance of your connection.

+ +

AT&T <%= link_to 'Run Test', '/ping/pingat.jnlp', :class=>'button' %>

+ +

Comcast <%= link_to 'Run Test', '/ping/pingcc.jnlp', :class=>'button' %>

+ +

Time Warner <%= link_to 'Run Test', '/ping/pingtw.jnlp', :class=>'button' %>

+ +

Verizon <%= link_to 'Run Test', '/ping/pingvz.jnlp', :class=>'button' %>

+ +

None Of The Above <%= link_to 'Run Test', '/ping/pingno.jnlp', :class=>'button' %>

+
\ No newline at end of file From 267664d2bf4c76b4325ac008150ee14ba4d3c5b0 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Tue, 17 Dec 2013 23:55:10 +0000 Subject: [PATCH 02/20] * adding in updated jnlp files --- ruby/spec/jam_ruby/models/user_spec.rb | 1 + web/app/assets/images/isps/ping-icon.jpg | Bin 0 -> 3595 bytes web/app/controllers/ping_controller.rb | 5 +++++ web/app/views/layouts/ping.jnlp.erb | 12 +++++++----- web/app/views/ping/pingat.jnlp.erb | 9 ++++++++- web/app/views/ping/pingcc.jnlp.erb | 9 ++++++++- web/app/views/ping/pingno.jnlp.erb | 9 ++++++++- web/app/views/ping/pingtw.jnlp.erb | 9 ++++++++- web/app/views/ping/pingvz.jnlp.erb | 10 +++++++++- web/config/routes.rb | 1 + 10 files changed, 55 insertions(+), 10 deletions(-) create mode 100644 web/app/assets/images/isps/ping-icon.jpg diff --git a/ruby/spec/jam_ruby/models/user_spec.rb b/ruby/spec/jam_ruby/models/user_spec.rb index df1c01ef7..65b387a1d 100644 --- a/ruby/spec/jam_ruby/models/user_spec.rb +++ b/ruby/spec/jam_ruby/models/user_spec.rb @@ -118,6 +118,7 @@ describe User do it "should be saved as all lower-case" do + pending @user.email = mixed_case_email @user.save! @user.reload.email.should == mixed_case_email diff --git a/web/app/assets/images/isps/ping-icon.jpg b/web/app/assets/images/isps/ping-icon.jpg new file mode 100644 index 0000000000000000000000000000000000000000..b1585aa94bcd589bdd519e9fc1c09f5b8d9eed71 GIT binary patch literal 3595 zcmbW2c{r49`^WFGWbC0qN>kP*51#B)lo(qjYnfpbp&?0SAEfL{cEW=sDNDjw$}-iU zP-$Z_W1F!(#l(>5X0~^{?|Z!Oas2-J{eI_tUB_`B$9-P+@%^0N>%1=BH{J}e(;jJu z1RxLyaDo2-JPL3U5P(3o$G<~RSa3Ub2nh)ai|h~)*`82QaZxB#3@RcbCMhN+F2N5G zQ7LIj390S(?VoHf-(JNZ5>OH7cEpx`wgG+|b zCBXMUNMxG}A`ro!f-*wF2M+I$J>evBF;q@PF9EvCGPkPvm8hz|^D4~e`X@1YH3Ol z-sIjvPFAqcl=nEhoEuPP5IItDZy?$zbRbrB#+WWSGeVA_{)$7aYV-url%?$E+V%Ug z<%oyirKYO6?zYpy=%B~yUW$Gmc3;8j)h;3rQ0j#T_QDwMg+YyAJo*-8%40&X#qI6> z+fcu-3k^>n>gm-ZBp~n#!ZYuex>**_MA*B`Sk&^TB~3r#1=;70&C>!MgVldlN;n9O zV6Ty&Y05o6FQ$vSCQ0Y6>2WGf%=sKPDOK!T*9mmNUT(`+UyJ1dwOes^Idi*5Bx%aK zUzpyN*L%FV&{G~ANPqn3X&utY9?=`T#j30^Tr;LWVTIsqIi-Z)kUJRYXS}Yszhm!Dt;N&V?@9_A^yRCy%uvBHwBsn1ALL_q?fSyL>!A2nVC-D&v&F+S!$K zt^(*r<=4{ufS2{tqbJnl$D$v$c%tl!78bBH+w+{9$JD{;9D0DSyfeITc2uX1 zwYg7Pz~_3YX-m|E>thqQGe4a1J%2q{avEd*IK;$iP+CS~?W_YL9s0@Vvf0|TL8bDw zwM!_C=3r%da@ED_#YP<;ZL4OB5N8W-`t7er@at;024~CfVO|B_v(85byo?hE{q?C? zmRk`F?~=zSuvf}3l;u3nt12EUmH{~r?b#2Skzv_;X6qfL?OihWKI|Qu`rW~{@clqN zRVH1wP;gDL;nbgB7vln3L7xK8vPrKl)S;!)(^~Zt%WO9uAbbtfuEvG05{U2N=ogGf z&uLli6kBw*w2ksqrLQ*eb*@3jA4_|l=u6y_sKIqez;3zx8pRE>WO=|VM6V*ZUm?77 zK2ZUq&Up8y!D*W0DfViKs#O8{IH4=yeEkf2EveK>ucaCGYt;AF7UfQn&R|d8-#yW- zpJvsGXZu`;t0VYA6XfS7&VyeIe`h*-Hn$HhO?r71E_mY!^ld-JQ%*GJJ2jJVHcdfD zzfaw!_sNZ8^$m(XdhdK~Zf2?8J^Xy3)RuPYL2Gg@tlN>hPm^(ytCYgsX#x6I?&ATk zP5FJ{$P{6#m;L1SGacJD;baq*)QHwFXJu=$D?S@;c$2@)d4L}wnb7V?hcvukJ2GJH zT~xZvQN(P#M5Ie!``AiMt*0HcGuTi$mDGNb>+jxPI%elI?on(R2$#0{_EVZt$$L$%I<9_fcaCVa|a19Wh^a>e7T<`KCyI&pq61J8?-!8rhr+WD) zZ1w6Qb)j)pY6Qcn4Rbhsgi1;YHt_Q9QxJbN61=Nr>BbLZIqVebb>8M9`7e8}`nu~D zYdOsPP7R9(H7eP#7%Kd=cbd687{v(b;{nPygWA4u52G%3w8!bD)5r*=AL8R7A%>#< z9ihtY-)1VNpeJ*3>JB{~04=EE*%UNaJc)7A6^xCda*iD5q_P#@3EaI{(oOl{0+<^%QDDYg5V2>4KwnY)gY;Ag6-L|I{BgG{^aE; zcyW=4IX-Kf@@PBe}tJtjZ~&F-CK-->q@SLilxqf`a_+eexv^ z${lgxdmIBxn(rYb{|XMDB^Jw*4IUh^y64$jbL|<$3*YrM_tJ0K-F}p3`Vh=3mKJ~G zp5n^Kjqw2SUe(BHtDaG|;mk@BnX{O((dgShCV3zIM*X4FV5qr{efQbglD6KD&1MEy zcPjv;d0C5ZR}`nc@TavlaSkhMmj+G`=-7X7Z`tT+pQgr#{*r{PFD`HL1=@n@EN7e_ z9*m!9k_{qpwZ0ligZ2${T^zg-8|sZcGAhsA{gbYej1CAmn^#?!Z*d@SLpqg!k06dD zDovn5GH_8dBjP{wU+Va_jc#-WA?8wB59X*nZE$Tdk<9cvA8sfYD6>$J_Ismap>)IM z%pP%XIhSwLnMIvXr+b=ONdA>ox{_dQei-bUe{YBr>{Qd$<0+9l%h?I(o7O z3_;$x#p9-gga3ww!v-gXEq_#N> z^dG?Ddwj&v)i zDxGQw-yK=j7meqtGrUt*Xm?FF-lOIYbXJN%&JXLpZTK)`*19h!n+M!}7WXDzju2c1 zPr~e3EvzM$JHEug8NnTCY`>bwYWL^OW;Bl+W>MO_hnJS6zxky)ig9Wo1Tw5k3Y&|^ zpQD#kDMUxnpfM%RR5ioJ%?z2Zo+x7{-2DK3x#JAR)z6FMw3faI>6{THr8Pm< z#_E@Wbt{~q* zVfZRxAU-X+@>avI_{1rR-c`PMI@{vd=(}RGHpcjkhT-6&2iuRo6wt04iO5_(;;T(F zm*VHhf<*oqbNdu_x^*QpP6Ldv9xWc2UM&H$UB4{rhN0mK!kyUp6xZn+Si_+STUqym zAEVF*j$it7b1g25?f7vcTaInWkSe3nQQ9KjUkxZ_!Nx9m6B8Qc*BN^KrkQ#Q(0TM_ zzYi0a!kRMMyuZC+rh=_3<0VH$3VIod#&#Y(O?2hcG3pl4b|DX(!fhAaoiocm_27zCZg)fs z->g`g!VQ;96|-eDLRrWkaCshZxui*Vqdr7AkP&I|n%u}KZ*1&mCOwGCQR8~GsM+RW z9AesBWmVI6zFV%z{tU-EvgH=3()5e7LEbWv{A0s{vh}_&9s6t|J`h7*?(k5{b-5y^ z^4HOpy}6ZTPk}dfEN$!6X)6$36&XP_uc4OPyfT++%dl)#g8ecs8p{x}6|$!JbiLl? zo8P%25yuSr_??FM*aX{AFw^+r43l*_IO@nCr9Qyl>+ZWXoI_*MU#lC1q%MS|ljHIv z%H0VcHmQbl;#+)(@6*-D73MEaRWIC2Hx<7Jo{UjSP$9JDMWnbQOBanlQ*wh<^xL|_ zsY2QKquw_K`g;&{k^N17 zH(s;AJ1+Ongpi4^jCz$B+%yiM+QVC;Mv_3$(GK_g<2thGg$UBb?vkP=-1t;N=DMd^ z^Lab!Y)~7?w~1JxSwrl$OZ*VDZsMGOeZDn6BO}9}uIpsJfFTjz$Q%jh~%xvL@4K z)tG*9rp#Lk JNLP end + def icon + redirect_to '/assets/isps/ping-icon.jpg' + #send_file Rails.root.join("app", "assets", "images", "isps", "ping-icon.jpg"), type: "image/jpg", disposition: "inline" + end + end diff --git a/web/app/views/layouts/ping.jnlp.erb b/web/app/views/layouts/ping.jnlp.erb index 3fc4fbeea..6c5325197 100644 --- a/web/app/views/layouts/ping.jnlp.erb +++ b/web/app/views/layouts/ping.jnlp.erb @@ -1,17 +1,19 @@ - Ping - JamKazam + JamKazam Ping + JamKazam, Inc. + + - + - + - da1-cc=50.242.148.38:4442 + <%= yield(:hosts) %> -u<%= ApplicationHelper.base_uri(request) %>/api/users/isp_scoring -i<%= yield(:provider) %> -a diff --git a/web/app/views/ping/pingat.jnlp.erb b/web/app/views/ping/pingat.jnlp.erb index da63cae0b..0e5040f1a 100755 --- a/web/app/views/ping/pingat.jnlp.erb +++ b/web/app/views/ping/pingat.jnlp.erb @@ -1 +1,8 @@ -<% provide(:provider, 'at') %> \ No newline at end of file +<% provide(:provider, 'at') %> + +<% content_for :hosts do %> + da1-vz=157.130.141.42:4442 + + + da2-at=12.251.184.250:4442 +<% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingcc.jnlp.erb b/web/app/views/ping/pingcc.jnlp.erb index ccc2d8e7d..73cfeafe5 100755 --- a/web/app/views/ping/pingcc.jnlp.erb +++ b/web/app/views/ping/pingcc.jnlp.erb @@ -1 +1,8 @@ -<% provide(:provider, 'cc') %> \ No newline at end of file +<% provide(:provider, 'cc') %> + +<% content_for :hosts do %> + da1-vz=157.130.141.42:4442 + + + da2-at=12.251.184.250:4442 +<% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingno.jnlp.erb b/web/app/views/ping/pingno.jnlp.erb index cd2a0b04e..949b81172 100755 --- a/web/app/views/ping/pingno.jnlp.erb +++ b/web/app/views/ping/pingno.jnlp.erb @@ -1 +1,8 @@ -<% provide(:provider, 'no') %> \ No newline at end of file +<% provide(:provider, 'no') %> + +<% content_for :hosts do %> + da1-vz=157.130.141.42:4442 + + + da2-at=12.251.184.250:4442 +<% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingtw.jnlp.erb b/web/app/views/ping/pingtw.jnlp.erb index e7d6a5b18..c4c64885b 100755 --- a/web/app/views/ping/pingtw.jnlp.erb +++ b/web/app/views/ping/pingtw.jnlp.erb @@ -1 +1,8 @@ -<% provide(:provider, 'tw') %> \ No newline at end of file +<% provide(:provider, 'tw') %> + +<% content_for :hosts do %> + da1-vz=157.130.141.42:4442 + + + da2-at=12.251.184.250:4442 +<% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingvz.jnlp.erb b/web/app/views/ping/pingvz.jnlp.erb index b31f5b3eb..28e7f014d 100755 --- a/web/app/views/ping/pingvz.jnlp.erb +++ b/web/app/views/ping/pingvz.jnlp.erb @@ -1 +1,9 @@ -<% provide(:provider, 'vz') %> \ No newline at end of file +<% provide(:provider, 'vz') %> + +<% content_for :hosts do %> + + da1-vz=157.130.141.42:4442 + + + da2-at=12.251.184.250:4442 +<% end %> \ No newline at end of file diff --git a/web/config/routes.rb b/web/config/routes.rb index 4fd9b3284..31ab9d0e7 100644 --- a/web/config/routes.rb +++ b/web/config/routes.rb @@ -51,6 +51,7 @@ SampleApp::Application.routes.draw do match '/ping/pingno.jnlp', to: 'ping#no' match '/ping/pingtw.jnlp', to: 'ping#tw' match '/ping/pingvz.jnlp', to: 'ping#vz' + match '/ping/icon.jpg', to:'ping#icon', :as => 'ping_icon' # spikes match '/facebook_invite', to: 'spikes#facebook_invite' From 5fee5e5f9cbb2f61b67d97f751ca6c4032ed83c7 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 18 Dec 2013 14:43:51 +0000 Subject: [PATCH 03/20] * touching gitignore to get another build to go off on master branch --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitignore b/.gitignore index 7393e2aa5..5e26ec47d 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ .idea *~ *.swp +web/screenshot*.html +web/screenshot*.png From dc5f5c48a87729cfd9125910a73d982af2e2c1c3 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 18 Dec 2013 15:47:02 +0000 Subject: [PATCH 04/20] * trying to fix admin based on stuff that broke in develop branch on Nov 5 --- admin/Gemfile | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/admin/Gemfile b/admin/Gemfile index 7d49040dd..6a249b8fc 100644 --- a/admin/Gemfile +++ b/admin/Gemfile @@ -33,21 +33,24 @@ group :assets do end gem 'will_paginate', '3.0.3' gem 'bootstrap-will_paginate', '0.0.6' -gem 'carrierwave' +gem 'carrierwave', '0.9.0' gem 'uuidtools', '2.1.2' gem 'bcrypt-ruby', '3.0.1' gem 'jquery-rails', '2.3.0' # pinned because jquery-ui-rails was split from jquery-rails, but activeadmin doesn't support this gem yet gem 'rails3-jquery-autocomplete' -gem 'activeadmin' -gem "meta_search", '>= 1.1.0.pre' -gem 'fog', "~> 1.3.1" +gem 'activeadmin', '0.6.2' +gem 'mime-types', '1.25' +gem 'meta_search' +gem 'fog', "~> 1.18.0" +gem 'unf', '0.1.3' #optional fog dependency gem 'country-select' gem 'aasm', '3.0.16' -gem 'postgres-copy' +gem 'postgres-copy', '0.6.0' gem 'aws-sdk' -gem 'bugsnag' +gem 'bugsnag' -gem 'eventmachine', '1.0.0' + +gem 'eventmachine', '1.0.3' gem 'amqp', '0.9.8' gem 'logging-rails', :require => 'logging/rails' @@ -56,6 +59,9 @@ gem 'ruby-protocol-buffers', '1.2.2' gem 'sendgrid', '1.1.0' +gem 'geokit-rails' +gem 'postgres_ext', '1.0.0' + group :libv8 do gem 'libv8', "~> 3.11.8" end From 69da7ffe1e030c3353148f0ef2d7c250018f60cc Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 18 Dec 2013 16:28:05 +0000 Subject: [PATCH 05/20] * uncommenting all ping servers --- web/app/views/ping/pingat.jnlp.erb | 4 ++-- web/app/views/ping/pingcc.jnlp.erb | 4 ++-- web/app/views/ping/pingno.jnlp.erb | 4 ++-- web/app/views/ping/pingtw.jnlp.erb | 4 ++-- web/app/views/ping/pingvz.jnlp.erb | 5 ++--- 5 files changed, 10 insertions(+), 11 deletions(-) diff --git a/web/app/views/ping/pingat.jnlp.erb b/web/app/views/ping/pingat.jnlp.erb index 0e5040f1a..4516b7177 100755 --- a/web/app/views/ping/pingat.jnlp.erb +++ b/web/app/views/ping/pingat.jnlp.erb @@ -2,7 +2,7 @@ <% content_for :hosts do %> da1-vz=157.130.141.42:4442 - - + da1-tw=50.84.4.230:4442 + da1-cc=50.242.148.38:4442 da2-at=12.251.184.250:4442 <% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingcc.jnlp.erb b/web/app/views/ping/pingcc.jnlp.erb index 73cfeafe5..17a5dfe36 100755 --- a/web/app/views/ping/pingcc.jnlp.erb +++ b/web/app/views/ping/pingcc.jnlp.erb @@ -2,7 +2,7 @@ <% content_for :hosts do %> da1-vz=157.130.141.42:4442 - - + da1-tw=50.84.4.230:4442 + da1-cc=50.242.148.38:4442 da2-at=12.251.184.250:4442 <% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingno.jnlp.erb b/web/app/views/ping/pingno.jnlp.erb index 949b81172..bb065e4ff 100755 --- a/web/app/views/ping/pingno.jnlp.erb +++ b/web/app/views/ping/pingno.jnlp.erb @@ -2,7 +2,7 @@ <% content_for :hosts do %> da1-vz=157.130.141.42:4442 - - + da1-tw=50.84.4.230:4442 + da1-cc=50.242.148.38:4442 da2-at=12.251.184.250:4442 <% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingtw.jnlp.erb b/web/app/views/ping/pingtw.jnlp.erb index c4c64885b..b5082b480 100755 --- a/web/app/views/ping/pingtw.jnlp.erb +++ b/web/app/views/ping/pingtw.jnlp.erb @@ -2,7 +2,7 @@ <% content_for :hosts do %> da1-vz=157.130.141.42:4442 - - + da1-tw=50.84.4.230:4442 + da1-cc=50.242.148.38:4442 da2-at=12.251.184.250:4442 <% end %> \ No newline at end of file diff --git a/web/app/views/ping/pingvz.jnlp.erb b/web/app/views/ping/pingvz.jnlp.erb index 28e7f014d..25344ffee 100755 --- a/web/app/views/ping/pingvz.jnlp.erb +++ b/web/app/views/ping/pingvz.jnlp.erb @@ -1,9 +1,8 @@ <% provide(:provider, 'vz') %> <% content_for :hosts do %> - da1-vz=157.130.141.42:4442 - - + da1-tw=50.84.4.230:4442 + da1-cc=50.242.148.38:4442 da2-at=12.251.184.250:4442 <% end %> \ No newline at end of file From f578a3b8ec68000d11d12dbfafd22733d62f6caa Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 18 Dec 2013 18:09:36 +0000 Subject: [PATCH 06/20] * accidentally left out the security 'all-permissions' check --- web/app/views/layouts/ping.jnlp.erb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/web/app/views/layouts/ping.jnlp.erb b/web/app/views/layouts/ping.jnlp.erb index 6c5325197..dc2c8adc1 100644 --- a/web/app/views/layouts/ping.jnlp.erb +++ b/web/app/views/layouts/ping.jnlp.erb @@ -19,4 +19,7 @@ -a + + + \ No newline at end of file From c0b5d63f5513b6466049d1d3cf9b41df0570a5b3 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 18 Dec 2013 21:58:11 +0000 Subject: [PATCH 07/20] * fixing steps typo --- web/app/views/ping/ping.html.erb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/web/app/views/ping/ping.html.erb b/web/app/views/ping/ping.html.erb index 728b9be04..d2e8dcda2 100644 --- a/web/app/views/ping/ping.html.erb +++ b/web/app/views/ping/ping.html.erb @@ -39,7 +39,7 @@
  • When you click the Run Test button, a file will start to download in your browser. It may display a message like “This type of file can harm your computer. Do you want to keep it anyway?”. Please click the button that lets your browser go ahead and download and save the file. It’s just a little test app we wrote ourselves, so we know it’s safe, but we did not sign the app with a certificate, so you may get this warning message.
    • -
  • When the file is downloaded, please click on the file to open and run it. If your version of Java is up-to-date, the app will run as described in step 7 below. If you get a message that Java is not up-to-date on your computer, please follow step 6 below to update Java on your computer.
    • +
  • When the file is downloaded, please click on the file to open and run it. If your version of Java is up-to-date, the app will run as described in step 8 below. If you get a message that Java is not up-to-date on your computer, please follow step 7 below to update Java on your computer.
  • Click the prompt button to update Java. You are taken to the Java website. Click the red Free Java Download button. Then click the Agree & Start Free Download button. When the file is downloaded, click on the file to open/run it, and then follow the on-screen instructions to install the Java update. (Note: Watch out during the installation for the McAfee option, and uncheck that one to avoid getting McAfee installed on your computer.) After the Java update has installed, go back to the JamKazam test app webpage, and click on the Run Test button again next to the ISP that provides Internet service to your home.
    • @@ -66,4 +66,4 @@

    None Of The Above <%= link_to 'Run Test', '/ping/pingno.jnlp', :class=>'button' %>

    - \ No newline at end of file + From 8262725aaebdba9fb46eef6fec333fbcf43c7251 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Mon, 28 Apr 2014 19:47:24 +0000 Subject: [PATCH 08/20] * merging in some gateway fixes --- web/app/assets/javascripts/JamServer.js | 14 +- web/app/assets/javascripts/configureTrack.js | 14 +- web/app/assets/javascripts/gear_wizard.js | 743 +++++++++++++++--- web/app/assets/javascripts/globals.js | 15 +- .../stylesheets/client/gearWizard.css.scss | 57 +- .../views/clients/gear/_gear_wizard.html.haml | 24 +- .../assets/javascripts/jquery.icheck.js | 4 +- .../lib/jam_websockets/router.rb | 23 +- 8 files changed, 738 insertions(+), 156 deletions(-) diff --git a/web/app/assets/javascripts/JamServer.js b/web/app/assets/javascripts/JamServer.js index d1126adf1..aaf2cfdd6 100644 --- a/web/app/assets/javascripts/JamServer.js +++ b/web/app/assets/javascripts/JamServer.js @@ -17,7 +17,8 @@ // heartbeat var heartbeatInterval = null; var heartbeatMS = null; - var heartbeatMissedMS = 10000; // if 5 seconds go by and we haven't seen a heartbeat ack, get upset + var heartbeatMissedMS = 10000; // if 10 seconds go by and we haven't seen a heartbeat ack, get upset + var lastHeartbeatSentTime = null; var lastHeartbeatAckTime = null; var lastHeartbeatFound = false; var heartbeatAckCheckInterval = null; @@ -140,6 +141,16 @@ var message = context.JK.MessageFactory.heartbeat(notificationLastSeen, notificationLastSeenAt); notificationLastSeenAt = undefined; notificationLastSeen = undefined; + // for debugging purposes, see if the last time we've sent a heartbeat is way off (500ms) of the target interval + var now = new Date(); + + if(lastHeartbeatSentTime) { + var drift = new Date().getTime() - lastHeartbeatSentTime.getTime() - heartbeatMS; + if(drift > 500) { + logger.error("significant drift between heartbeats: " + drift + 'ms beyond target interval') + } + } + lastHeartbeatSentTime = now; context.JK.JamServer.send(message); lastHeartbeatFound = false; } @@ -417,6 +428,7 @@ connectTimeout = setTimeout(function() { connectTimeout = null; if(connectDeferred.state() === 'pending') { + server.close(true); connectDeferred.reject(); } }, 4000); diff --git a/web/app/assets/javascripts/configureTrack.js b/web/app/assets/javascripts/configureTrack.js index d3bfb0eb2..344c99302 100644 --- a/web/app/assets/javascripts/configureTrack.js +++ b/web/app/assets/javascripts/configureTrack.js @@ -7,18 +7,8 @@ var logger = context.JK.logger; var myTrackCount; - var ASSIGNMENT = { - CHAT: -2, - OUTPUT: -1, - UNASSIGNED: 0, - TRACK1: 1, - TRACK2: 2 - }; - - var VOICE_CHAT = { - NO_CHAT: "0", - CHAT: "1" - }; + var ASSIGNMENT = context.JK.ASSIGNMENT; + var VOICE_CHAT = context.JK.VOICE_CHAT; var instrument_array = []; diff --git a/web/app/assets/javascripts/gear_wizard.js b/web/app/assets/javascripts/gear_wizard.js index 4926c6777..9012c91be 100644 --- a/web/app/assets/javascripts/gear_wizard.js +++ b/web/app/assets/javascripts/gear_wizard.js @@ -6,18 +6,27 @@ context.JK = context.JK || {}; context.JK.GearWizard = function (app) { + var ASSIGNMENT = context.JK.ASSIGNMENT; + var VOICE_CHAT = context.JK.VOICE_CHAT; + var $dialog = null; var $wizardSteps = null; var $currentWizardStep = null; var step = 0; var $templateSteps = null; var $templateButtons = null; + var $templateAudioPort = null; var $ftueButtons = null; var self = null; var operatingSystem = null; - // SELECT DEVICE STATE - var validScore = false; + // populated by loadDevices + var deviceInformation = null; + var musicPorts = null; + + + var validLatencyScore = false; + var validIOScore = false; // SELECT TRACKS STATE @@ -30,20 +39,33 @@ var STEP_ROUTER_NETWORK = 5; var STEP_SUCCESS = 6; + var PROFILE_DEV_SEP_TOKEN = '^'; + + var iCheckIgnore = false; + var audioDeviceBehavior = { - MacOSX_builtin : { + MacOSX_builtin: { + display: 'MacOSX Built-In', videoURL: undefined }, - MACOSX_interface : { + MacOSX_interface: { + display: 'MacOSX external interface', videoURL: undefined }, - Win32_wdm : { + Win32_wdm: { + display: 'Windows WDM', videoURL: undefined }, - Win32_asio : { + Win32_asio: { + display: 'Windows ASIO', videoURL: undefined }, - Win32_asio4all : { + Win32_asio4all: { + display: 'Windows ASIO4ALL', + videoURL: undefined + }, + Linux: { + display: 'Linux', videoURL: undefined } } @@ -51,7 +73,7 @@ function beforeShowIntro() { var $watchVideo = $currentWizardStep.find('.watch-video'); var videoUrl = 'https://www.youtube.com/watch?v=VexH4834o9I'; - if(operatingSystem == "Win32") { + if (operatingSystem == "Win32") { $watchVideo.attr('href', 'https://www.youtube.com/watch?v=VexH4834o9I'); } $watchVideo.attr('href', videoUrl); @@ -65,12 +87,90 @@ var $audioOutput = $currentWizardStep.find('.select-audio-output-device'); var $bufferIn = $currentWizardStep.find('.select-buffer-in'); var $bufferOut = $currentWizardStep.find('.select-buffer-out'); - var $nextButton = $ftueButtons.find('.btn-next'); var $frameSize = $currentWizardStep.find('.select-frame-size'); + var $inputChannels = $currentWizardStep.find('.input-ports'); + var $outputChannels = $currentWizardStep.find('.output-ports'); + var $scoreReport = $currentWizardStep.find('.results'); + var $latencyScoreSection = $scoreReport.find('.latency-score-section'); + var $latencyScore = $scoreReport.find('.latency-score'); + var $ioScoreSection = $scoreReport.find('.io-score-section'); + var $ioRateScore = $scoreReport.find('.io-rate-score'); + var $ioVarScore = $scoreReport.find('.io-var-score'); + var $ioCountdown = $scoreReport.find('.io-countdown'); + var $ioCountdownSecs = $scoreReport.find('.io-countdown .secs'); + var $nextButton = $ftueButtons.find('.btn-next'); + var $asioControlPanelBtn = $currentWizardStep.find('.asio-settings-btn'); + var $resyncBtn = $currentWizardStep.find('resync-btn') - // returns a deviceInfo hash for the device matching the deviceId, or null. + // should return one of: + // * MacOSX_builtin + // * MACOSX_interface + // * Win32_wdm + // * Win32_asio + // * Win32_asio4all + // * Linux + function determineDeviceType(deviceId, displayName) { + if (operatingSystem == "MacOSX") { + if (displayName.toLowerCase().trim() == "built-in") { + return "MacOSX_builtin"; + } + else { + return "MacOSX_interface"; + } + } + else if (operatingSystem == "Win32") { + if (context.jamClient.FTUEIsMusicDeviceWDM(deviceId)) { + return "Win32_wdm"; + } + else if (displayName.toLowerCase().indexOf("asio4all") > -1) { + return "Win32_asio4all" + } + else { + return "Win32_asio"; + } + } + else { + return "Linux"; + } + } + + function loadDevices() { + + var oldDevices = context.jamClient.FTUEGetDevices(false); + var devices = context.jamClient.FTUEGetAudioDevices(); + console.log("oldDevices: " + JSON.stringify(oldDevices)); + console.log("devices: " + JSON.stringify(devices)); + + var loadedDevices = {}; + + // augment these devices by determining their type + context._.each(devices.devices, function (device) { + + if(device.name == "JamKazam Virtual Monitor") { + return; + } + + var deviceInfo = {}; + + deviceInfo.id = device.guid; + deviceInfo.type = determineDeviceType(device.guid, device.display_name); + console.log("deviceInfo.type: " + deviceInfo.type) + deviceInfo.displayType = audioDeviceBehavior[deviceInfo.type].display; + deviceInfo.displayName = device.display_name; + + loadedDevices[device.guid] = deviceInfo; + + logger.debug("loaded device: ", deviceInfo); + }) + + deviceInformation = loadedDevices; + + logger.debug(context.JK.dlen(deviceInformation) + " devices loaded.", deviceInformation); + } + + // returns a deviceInfo hash for the device matching the deviceId, or undefined. function findDevice(deviceId) { - return {}; + return deviceInformation[deviceId]; } function selectedAudioInput() { @@ -81,31 +181,369 @@ return $audioOutput.val(); } + function selectedFramesize() { + return parseFloat($frameSize.val()); + } + + function selectedBufferIn() { + return parseFloat($frameSize.val()); + } + + function selectedBufferOut() { + return parseFloat($frameSize.val()); + } + function initializeNextButtonState() { $nextButton.removeClass('button-orange button-grey'); - if(validScore) $nextButton.addClass('button-orange'); + if (validLatencyScore) $nextButton.addClass('button-orange'); else $nextButton.addClass('button-grey'); } - function audioDeviceUnselected() { - validScore = false; + function initializeAudioInput() { + var optionsHtml = ''; + optionsHtml = ''; + context._.each(deviceInformation, function (deviceInfo, deviceId) { + + console.log(arguments) + optionsHtml += ''; + }); + $audioInput.html(optionsHtml); + context.JK.dropdown($audioInput); + + initializeAudioInputChanged(); + } + + function initializeAudioOutput() { + var optionsHtml = ''; + optionsHtml = ''; + context._.each(deviceInformation, function (deviceInfo, deviceId) { + optionsHtml += ''; + }); + $audioOutput.html(optionsHtml); + context.JK.dropdown($audioOutput); + + initializeAudioOutputChanged(); + } + + function initializeFramesize() { + context.JK.dropdown($frameSize); + } + + function initializeBuffers() { + context.JK.dropdown($bufferIn); + context.JK.dropdown($bufferOut); + } + // reloads the backend's channel state for the currently selected audio devices, + // and update's the UI accordingly + function initializeChannels() { + musicPorts = jamClient.FTUEGetChannels(); + console.log("musicPorts: %o", JSON.stringify(musicPorts)); + + initializeInputPorts(musicPorts); + initializeOutputPorts(musicPorts); + } + + // during this phase of the FTUE, we have to assign selected input channels + // to tracks. The user, however, does not have a way to indicate which channel + // goes to which track (that's not until the next step of the wizard). + // so, we just auto-generate a valid assignment + function newInputAssignment() { + var assigned = 0; + context._.each(musicPorts.inputs, function(inputChannel) { + if(isChannelAssigned(inputChannel)) { + assigned += 1; + } + }); + + var newAssignment = Math.floor(assigned / 2) + 1; + return newAssignment; + } + + function inputChannelChanged() { + if(iCheckIgnore) return; + + var $checkbox = $(this); + var channelId = $checkbox.attr('data-id'); + var isChecked = $checkbox.is(':checked'); + + if(isChecked) { + var newAssignment = newInputAssignment(); + logger.debug("assigning input channel %o to track: %o", channelId, newAssignment); + context.jamClient.TrackSetAssignment(channelId, true, newAssignment); + } + else { + logger.debug("unassigning input channel %o", channelId); + context.jamClient.TrackSetAssignment(channelId, true, ASSIGNMENT.UNASSIGNED); + // unassigning creates a hole in our auto-assigned tracks. reassign them all to keep it consistent + var $assignedInputs = $inputChannels.find('input[type="checkbox"]:checked'); + var assigned = 0; + context._.each($assignedInputs, function(assignedInput) { + var $assignedInput = $(assignedInput); + var assignedChannelId = $assignedInput.attr('data-id'); + var newAssignment = Math.floor(assigned / 2) + 1; + logger.debug("re-assigning input channel %o to track: %o", assignedChannelId, newAssignment); + context.jamClient.TrackSetAssignment(assignedChannelId, true, newAssignment); + assigned += 1; + }); + } + + initializeChannels(); + } + + // should be called in a ifChanged callback if you want to cancel. + // you have to use this instead of 'return false' like a typical input 'change' event. + function cancelICheckChange($checkbox) { + iCheckIgnore = true; + var checked = $checkbox.is(':checked'); + setTimeout(function() { + if(checked) $checkbox.iCheck('uncheck').removeAttr('checked'); + else $checkbox.iCheck('check').attr('checked', 'checked'); + iCheckIgnore = false; + }, 1); + } + + function outputChannelChanged() { + if(iCheckIgnore) return; + var $checkbox = $(this); + var channelId = $checkbox.attr('data-id'); + var isChecked = $checkbox.is(':checked'); + + // don't allow more than 2 output channels selected at once + if($outputChannels.find('input[type="checkbox"]:checked').length > 2) { + context.JK.Banner.showAlert('You can only have a maximum of 2 output ports selected.'); + // can't allow uncheck of last output + cancelICheckChange($checkbox); + return; + } + + if(isChecked) { + logger.debug("assigning output channel %o", channelId); + context.jamClient.TrackSetAssignment(channelId, true, ASSIGNMENT.OUTPUT); + } + else { + logger.debug("unassigning output channel %o", channelId); + context.jamClient.TrackSetAssignment(channelId, true, ASSIGNMENT.UNASSIGNED); + } + + initializeChannels(); + } + + // checks if it's an assigned OUTPUT or ASSIGNED CHAT + function isChannelAssigned(channel) { + return channel.assignment == ASSIGNMENT.CHAT || channel.assignment == ASSIGNMENT.OUTPUT || channel.assignment > 0; + } + + function initializeInputPorts(musicPorts) { + $inputChannels.empty(); + var inputPorts = musicPorts.inputs; + context._.each(inputPorts, function(inputChannel) { + var $inputChannel = $(context._.template($templateAudioPort.html(), inputChannel, { variable: 'data' })); + var $checkbox = $inputChannel.find('input'); + if(isChannelAssigned(inputChannel)) { + $checkbox.attr('checked', 'checked'); + } + context.JK.checkbox($checkbox); + $checkbox.on('ifChanged', inputChannelChanged); + $inputChannels.append($inputChannel); + }); + } + + function initializeOutputPorts(musicPorts) { + $outputChannels.empty(); + var outputChannels = musicPorts.outputs; + context._.each(outputChannels, function(outputChannel) { + var $outputPort = $(context._.template($templateAudioPort.html(), outputChannel, { variable: 'data' })); + var $checkbox = $outputPort.find('input'); + if(isChannelAssigned(outputChannel)) { + $checkbox.attr('checked', 'checked'); + } + context.JK.checkbox($checkbox); + $checkbox.on('ifChanged', outputChannelChanged); + $outputChannels.append($outputPort); + }); + } + + function initializeFormElements() { + if (!deviceInformation) throw "devices are not initialized"; + + initializeAudioInput(); + initializeAudioOutput(); + initializeFramesize(); + initializeBuffers(); + } + + function resetFrameBuffers() { + $frameSize.val('2.5'); + $bufferIn.val('0'); + $bufferOut.val('0'); + } + + function clearInputPorts() { + $inputChannels.empty(); + } + + function clearOutputPorts() { + $outputChannels.empty(); + } + + function resetScoreReport() { + $ioRateScore.empty(); + $ioVarScore.empty(); + $latencyScore.empty(); + } + + function renderLatencyScore(latencyValue, latencyClass) { + if(latencyValue) { + $latencyScore.text(latencyValue + ' ms'); + } + else { + $latencyScore.text(''); + } + $latencyScoreSection.removeClass('good acceptable bad unknown starting').addClass(latencyClass); + } + + // std deviation is the worst value between in/out + // media is the worst value between in/out + // io is the value returned by the backend, which has more info + // ioClass is the pre-computed rollup class describing the result in simple terms of 'good', 'acceptable', bad' + function renderIOScore(std, median, ioData, ioClass) { + $ioRateScore.text(median ? median : ''); + $ioVarScore.text(std ? std : ''); + $ioScoreSection.removeClass('good acceptable bad unknown starting skip').addClass(ioClass); + // TODO: show help bubble of all data in IO data + } + + function updateScoreReport(latencyResult) { + var latencyClass = "neutral"; + var latencyValue = 'N/A'; + var validLatency = false; + if (latencyResult && latencyResult.latencyknown) { + var latencyValue = latencyResult.latency; + latencyValue = Math.round(latencyValue * 100) / 100; + if (latencyValue <= 10) { + latencyClass = "good"; + validLatency = true; + } else if (latencyValue <= 20) { + latencyClass = "acceptable"; + validLatency = true; + } else { + latencyClass = "bad"; + } + } + else { + latencyClass = 'unknown'; + } + + validLatencyScore = validLatency; + + renderLatencyScore(latencyValue, latencyClass); + } + + function audioInputDeviceUnselected() { + validLatencyScore = false; initializeNextButtonState(); + resetFrameBuffers(); + clearInputPorts(); + } + + function renderScoringStarted() { + validLatencyScore = false; + initializeNextButtonState(); + resetScoreReport(); + freezeAudioInteraction(); + renderLatencyScore(null, 'starting'); + } + + function renderScoringStopped() { + initializeNextButtonState(); + unfreezeAudioInteraction(); + } + + + function freezeAudioInteraction() { + $audioInput.attr("disabled", "disabled").easyDropDown('disable'); + $audioOutput.attr("disabled", "disabled").easyDropDown('disable'); + $frameSize.attr("disabled", "disabled").easyDropDown('disable'); + $bufferIn.attr("disabled", "disabled").easyDropDown('disable'); + $bufferOut.attr("disabled", "disabled").easyDropDown('disable'); + $asioControlPanelBtn.on("click", false); + $resyncBtn.on('click', false); + iCheckIgnore = true; + $inputChannels.find('input[type="checkbox"]').iCheck('disable'); + $outputChannels.find('input[type="checkbox"]').iCheck('disable'); + } + + function unfreezeAudioInteraction() { + $audioInput.removeAttr("disabled").easyDropDown('enable'); + $audioOutput.removeAttr("disabled").easyDropDown('enable'); + $frameSize.removeAttr("disabled").easyDropDown('enable'); + $bufferIn.removeAttr("disabled").easyDropDown('enable'); + $bufferOut.removeAttr("disabled").easyDropDown('enable'); + $asioControlPanelBtn.off("click", false); + $resyncBtn.off('click', false); + $inputChannels.find('input[type="checkbox"]').iCheck('enable'); + $outputChannels.find('input[type="checkbox"]').iCheck('enable'); + iCheckIgnore = false; + } + + // Given a latency structure, update the view. + function newFtueUpdateLatencyView(latency) { + var $report = $('.ftue-new .latency .report'); + var $instructions = $('.ftue-new .latency .instructions'); + var latencyClass = "neutral"; + var latencyValue = "N/A"; + var $saveButton = $('#btn-ftue-2-save'); + if (latency && latency.latencyknown) { + latencyValue = latency.latency; + // Round latency to two decimal places. + latencyValue = Math.round(latencyValue * 100) / 100; + if (latency.latency <= 10) { + latencyClass = "good"; + setSaveButtonState($saveButton, true); + } else if (latency.latency <= 20) { + latencyClass = "acceptable"; + setSaveButtonState($saveButton, true); + } else { + latencyClass = "bad"; + setSaveButtonState($saveButton, false); + } + } else { + latencyClass = "unknown"; + setSaveButtonState($saveButton, false); + } + + $('.ms-label', $report).html(latencyValue); + $('p', $report).html('milliseconds'); + + $report.removeClass('good acceptable bad unknown'); + $report.addClass(latencyClass); + + var instructionClasses = ['neutral', 'good', 'acceptable', 'unknown', 'bad', 'start', 'loading']; + $.each(instructionClasses, function (idx, val) { + $('p.' + val, $instructions).hide(); + }); + if (latency === 'loading') { + $('p.loading', $instructions).show(); + } else { + $('p.' + latencyClass, $instructions).show(); + renderStopNewFtueLatencyTesting(); + } } function initializeWatchVideo() { - $watchVideoInput.unbind('click').click(function() { + $watchVideoInput.unbind('click').click(function () { var audioDevice = findDevice(selectedAudioInput()); - if(!audioDevice) { + if (!audioDevice) { context.JK.Banner.showAlert('You must first choose an Audio Input Device so that we can determine which video to show you.'); } else { - var videoURL = audioDeviceBehavior[audioDevice.type]; + var videoURL = audioDeviceBehavior[audioDevice.type].videoURL; - if(videoURL) { + if (videoURL) { $(this).attr('href', videoURL); return true; } @@ -117,16 +555,16 @@ return false; }); - $watchVideoOutput.unbind('click').click(function() { + $watchVideoOutput.unbind('click').click(function () { var audioDevice = findDevice(selectedAudioOutput()); - if(!audioDevice) { + if (!audioDevice) { throw "this button should be hidden"; } else { - var videoURL = audioDeviceBehavior[audioDevice.type]; + var videoURL = audioDeviceBehavior[audioDevice.type].videoURL; - if(videoURL) { + if (videoURL) { $(this).attr('href', videoURL); return true; } @@ -139,106 +577,131 @@ }); } - function initializeAudioInputChanged() { - $audioInput.unbind('change').change(function(evt) { - - var audioDeviceId = selectedAudioInput(); - - if(!audioDeviceId) { - audioDeviceUnselected(); - return false; - } - - var audioDevice = findDevice(selectedAudioInput()); - - if(!audioDevice) { - context.JK.alertSupportedNeeded('Unable to find device information for: ' + audioDevice); - } - - releaseDropdown(function () { - renderStartNewFtueLatencyTesting(); - var $select = $(evt.currentTarget); - - var $audioSelect = $('.ftue-new .settings-2-device select'); - var audioDriverId = $audioSelect.val(); - - if (!audioDriverId) { - context.JK.alertSupportNeeded(); - - renderStopNewFtueLatencyTesting(); - // reset back to 'Choose...' - newFtueEnableControls(false); - return; - } - jamClient.FTUESetMusicDevice(audioDriverId); - - var musicInputs = jamClient.FTUEGetMusicInputs(); - var musicOutputs = jamClient.FTUEGetMusicOutputs(); - - // set the music input to the first available input, - // and output to the first available output - var kin = null, kout = null, k = null; - // TODO FIXME - this jamClient call returns a dictionary. - // It's difficult to know what to auto-choose. - // For example, with my built-in audio, the keys I get back are - // digital in, line in, mic in and stereo mix. Which should we pick for them? - for (k in musicInputs) { - kin = k; - break; - } - for (k in musicOutputs) { - kout = k; - break; - } - var result; - if (kin && kout) { - jamClient.FTUESetMusicInput(kin); - jamClient.FTUESetMusicOutput(kout); - } else { - // TODO FIXME - how to handle a driver selection where we are unable to - // autoset both inputs and outputs? (I'd think this could happen if either - // the input or output side returned no values) - renderStopNewFtueLatencyTesting(); - console.log("invalid kin/kout %o/%o", kin, kout); - return; - } - - newFtueUpdateLatencyView('loading'); - - newFtueEnableControls(true); - newFtueOsSpecificSettings(); - // make sure whatever the user sees in the frontend is what the backend thinks - // this is necesasry because if you do a FTUE pass, close the client, and pick the same device - // the backend will *silently* use values from before, because the frontend does not query the backend - // for these values anywhere. - - // setting all 3 of these cause 3 FTUE's. Instead, we set batchModify to true so that they don't call FtueSave(false), and call later ourselves - batchModify = true; - newFtueAsioFrameSizeToBackend($('#ftue-2-asio-framesize')); - newFtueAsioInputLatencyToBackend($('#ftue-2-asio-input-latency')); - newFtueAsioOutputLatencyToBackend($('#ftue-2-asio-output-latency')); - batchModify = false; - - //setLevels(0); - renderVolumes(); - - logger.debug("Calling FTUESave(" + false + ")"); - jamClient.FTUESave(false) - pendingFtueSave = false; // this is not really used in any real fashion. just setting back to false due to batch modify above - - setVuCallbacks(); - - var latency = jamClient.FTUEGetExpectedLatency(); - console.log("FTUEGetExpectedLatency: %o", latency); - newFtueUpdateLatencyView(latency); - }); - - }) + function renderIOScoringStarted(secondsLeft) { + $ioCountdownSecs.text(secondsLeft); + $ioCountdown.show(); } + + function renderIOScoringStopped() { + $ioCountdown.hide(); + } + + function renderIOCountdown(secondsLeft) { + $ioCountdownSecs.text(secondsLeft); + } + + function attemptScore() { + var audioInputDeviceId = selectedAudioInput(); + var audioOutputDeviceId = selectedAudioOutput(); + if (!audioInputDeviceId) { + audioInputDeviceUnselected(); + return false; + } + + var audioInputDevice = findDevice(audioInputDeviceId); + if (!audioInputDevice) { + context.JK.alertSupportedNeeded('Unable to find information for input device: ' + audioInputDeviceId); + return false; + } + + if(!audioOutputDeviceId) { + audioOutputDeviceId = audioInputDeviceId; + } + var audioOutputDevice = findDevice(audioOutputDeviceId); + if (!audioInputDevice) { + context.JK.alertSupportedNeeded('Unable to find information for output device: ' + audioOutputDeviceId); + return false; + } + + jamClient.FTUESetInputMusicDevice(audioInputDeviceId); + jamClient.FTUESetOutputMusicDevice(audioOutputDeviceId); + + initializeChannels(); + + jamClient.FTUESetInputLatency(selectedBufferIn()); + jamClient.FTUESetOutputLatency(selectedBufferOut()); + jamClient.FTUESetFrameSize(selectedFramesize()); + + renderScoringStarted(); + logger.debug("Calling FTUESave(false)"); + jamClient.FTUESave(false); + + var latency = jamClient.FTUEGetExpectedLatency(); + console.log("FTUEGetExpectedLatency: %o", latency); + + updateScoreReport(latency); + + // if there was a valid latency score, go on to the next step + if(validLatencyScore) { + renderIOScore(null, null, null, 'starting'); + var testTimeSeconds = 10; // allow 10 seconds for IO to establish itself + context.jamClient.FTUEStartIoPerfTest(); + renderIOScoringStarted(testTimeSeconds); + renderIOCountdown(testTimeSeconds); + var interval = setInterval(function() { + testTimeSeconds -= 1; + renderIOCountdown(testTimeSeconds); + if(testTimeSeconds == 0) { + clearInterval(interval); + renderIOScoringStopped(); + var io = context.jamClient.FTUEGetIoPerfData(); + + console.log("io: ", io); + + // take the higher variance, which is apparently actually std dev + var std = io.in_var > io.out_var ? io.in_var : io.out_var; + std = Math.round(std * 100) / 100; + // take the furthest-off-from-target io rate + var median = Math.abs(io.in_median - io.in_target ) > Math.abs(io.out_median - io.out_target ) ? [io.in_median, io.in_target] : [io.out_median, io.out_target]; + var medianTarget = median[1]; + median = Math.round(median[0]); + + var stdIOClass = 'bad'; + if(std <= 0.50) { + stdIOClass = 'good'; + } + else if(std <= 1.00) { + stdIOClass = 'acceptable'; + } + + var medianIOClass = 'bad'; + if(Math.abs(median - medianTarget) <= 1) { + medianIOClass = 'good'; + } + else if(Math.abs(median - medianTarget) <= 2) { + medianIOClass = 'acceptable'; + } + + // now base the overall IO score based on both values. + renderIOScore(std, median, io, ioClass); + + // lie for now until IO questions finalize + validIOScore = true; + + renderScoringStopped(); + } + }, 1000); + } + else { + renderIOScore(null, null, null, 'skip'); + renderScoringStopped(); + } + + } + + function initializeAudioInputChanged() { + $audioInput.unbind('change').change(attemptScore); + } + + function initializeAudioOutputChanged() { + + } + function initializeStep() { + loadDevices(); + initializeFormElements(); initializeNextButtonState(); initializeWatchVideo(); - initializeAudioInputChanged(); } initializeStep(); @@ -291,7 +754,9 @@ function beforeShowStep($step) { var stepInfo = STEPS[step]; - if(!stepInfo) {throw "unknown step: " + step;} + if (!stepInfo) { + throw "unknown step: " + step; + } stepInfo.beforeShow.call(self); } @@ -304,7 +769,7 @@ $currentWizardStep = $nextWizardStep; var $ftueSteps = $(context._.template($templateSteps.html(), {}, { variable: 'data' })); - var $activeStep = $ftueSteps.find('.ftue-stepnumber[data-step-number="'+ step +'"]'); + var $activeStep = $ftueSteps.find('.ftue-stepnumber[data-step-number="' + step + '"]'); $activeStep.addClass('.active'); $activeStep.next().show(); // show the .ftue-step-title $currentWizardStep.find('.ftuesteps').replaceWith($ftueSteps); @@ -321,11 +786,11 @@ var $btnCancel = $ftueButtonsContent.find('.btn-cancel'); // hide back button if 1st step or last step - if(step == 0 && step == TOTAL_STEPS - 1) { + if (step == 0 && step == TOTAL_STEPS - 1) { $btnBack.hide(); } // hide next button if not on last step - if (step == TOTAL_STEPS - 1 ) { + if (step == TOTAL_STEPS - 1) { $btnNext.hide(); } // hide close if on last step @@ -350,9 +815,33 @@ $currentWizardStep = null; } + // checks if we already have a profile called 'FTUE...'; if not, create one. if so, re-use it. + function findOrCreateFTUEProfile() { + var profileName = context.jamClient.FTUEGetMusicProfileName(); + + logger.debug("current profile name: " + profileName); + + if(profileName && profileName.indexOf('FTUE') == 0) { + + } + else { + var newProfileName = 'FTUEAttempt-' + new Date().getTime().toString(); + logger.debug("setting FTUE-prefixed profile name to: " + newProfileName); + context.jamClient.FTUESetMusicProfileName(newProfileName); + } + + var profileName = context.jamClient.FTUEGetMusicProfileName(); + + logger.debug("name on exit: " + profileName); + + } + function beforeShow(args) { + context.jamClient.FTUECancel(); + findOrCreateFTUEProfile(); + step = args.d1; - if(!step) step = 0; + if (!step) step = 0; step = parseInt(step); moveToStep(); } @@ -362,18 +851,18 @@ } function afterHide() { - + context.jamClient.FTUECancel(); } function back() { - if($(this).is('.button-grey')) return; + if ($(this).is('.button-grey')) return; step = step - 1; moveToStep(); return false; } function next() { - if($(this).is('.button-grey')) return; + if ($(this).is('.button-grey')) return; step = step + 1; @@ -392,6 +881,7 @@ function route() { } + function initialize() { var dialogBindings = { beforeShow: beforeShow, afterShow: afterShow, afterHide: afterHide }; @@ -402,6 +892,7 @@ $wizardSteps = $dialog.find('.wizard-step'); $templateSteps = $('#template-ftuesteps'); $templateButtons = $('#template-ftue-buttons'); + $templateAudioPort = $('#template-audio-port'); $ftueButtons = $dialog.find('.ftue-buttons'); operatingSystem = context.jamClient.GetOSAsString(); diff --git a/web/app/assets/javascripts/globals.js b/web/app/assets/javascripts/globals.js index 49341c071..7c3a37ec3 100644 --- a/web/app/assets/javascripts/globals.js +++ b/web/app/assets/javascripts/globals.js @@ -14,7 +14,20 @@ UNIX: "Unix" }; - // TODO: store these client_id values in instruments table, or store + context.JK.ASSIGNMENT = { + CHAT: -2, + OUTPUT: -1, + UNASSIGNED: 0, + TRACK1: 1, + TRACK2: 2 + }; + + context.JK.VOICE_CHAT = { + NO_CHAT: "0", + CHAT: "1" + }; + + // TODO: store these client_id values in instruments table, or store // server_id as the client_id to prevent maintenance nightmares. As it's // set up now, we will have to deploy each time we add new instruments. context.JK.server_to_client_instrument_map = { diff --git a/web/app/assets/stylesheets/client/gearWizard.css.scss b/web/app/assets/stylesheets/client/gearWizard.css.scss index 49dc7f34b..aa2ac4bf6 100644 --- a/web/app/assets/stylesheets/client/gearWizard.css.scss +++ b/web/app/assets/stylesheets/client/gearWizard.css.scss @@ -184,8 +184,63 @@ width:45%; } + + .buffers { + .easydropdown-wrapper:nth-of-type(1) { + left:5px; + } + .easydropdown-wrapper:nth-of-type(2) { + left:35px; + } + .easydropdown, .easydropdown-wrapper { + width:15px; + } + } + + + .ftue-box.results { height: 230px !important; + padding:0; + + .scoring-section { + font-size:15px; + @include border_box_sizing; + height:64px; + + &.good { + background-color:#72a43b; + } + &.acceptable { + background-color:#cc9900; + } + &.bad, &.skip { + background-color:#660000; + } + &.unknown { + background-color:#999; + } + } + + .io-countdown { + display:none; + padding-left:19px; + position:relative; + + .secs { + position:absolute; + width:19px; + left:0; + } + } + + .io-skip-msg { + display:none; + + .scoring-section.skip & { + display:inline; + } + } } .audio-output { @@ -510,8 +565,8 @@ .subcolumn.third { right:0px; } - } + .settings-controls { clear:both; diff --git a/web/app/views/clients/gear/_gear_wizard.html.haml b/web/app/views/clients/gear/_gear_wizard.html.haml index 09d99a642..57f0ef1f8 100644 --- a/web/app/views/clients/gear/_gear_wizard.html.haml +++ b/web/app/views/clients/gear/_gear_wizard.html.haml @@ -38,7 +38,7 @@ %select.w100.select-audio-input-device %option None %h2 Audio Input Ports - .ftue-box.list.ports.output-ports + .ftue-box.list.ports.input-ports %a.button-orange.asio-settings-btn ASIO SETTINGS... %a.button-orange.resync-btn RESYNC .wizard-step-column @@ -46,7 +46,7 @@ %select.w100.select-audio-output-device %option Same as input %h2 Audio Output Ports - .ftue-box.list.ports.input-ports + .ftue-box.list.ports.output-ports .frame-and-buffers .framesize %h2 Frame @@ -83,6 +83,20 @@ .wizard-step-column %h2 Test Results .ftue-box.results + .left.w50.center.white.scoring-section.latency-score-section + .p5 + .latency LATENCY + %span.latency-score + .left.w50.center.white.scoring-section.io-score-section + .p5 + .io I/O + %span.io-skip-msg + Skipped + %span.io-countdown + %span.secs + seconds left + %span.io-rate-score + %span.io-var-score .clearall @@ -195,5 +209,11 @@ %a.button-orange.btn-next{href:'#'} NEXT %a.button-orange.btn-close{href:'#', 'layout-action' => 'close'} CLOSE +%script{type: 'text/template', id: 'template-audio-port'} + .audio-port + %input{ type: 'checkbox', 'data-id' => '{{data.id}}' } + %span + = '{{data.name}}' + diff --git a/web/vendor/assets/javascripts/jquery.icheck.js b/web/vendor/assets/javascripts/jquery.icheck.js index d7d819da3..4e0cffeb9 100644 --- a/web/vendor/assets/javascripts/jquery.icheck.js +++ b/web/vendor/assets/javascripts/jquery.icheck.js @@ -285,10 +285,12 @@ // Check, disable or indeterminate if (/^(ch|di|in)/.test(method) && !active) { + console.log("TAKING ROUTE: ", state); on(input, state); // Uncheck, enable or determinate } else if (/^(un|en|de)/.test(method) && active) { + console.log("TAKING ROUTE2: ", state); off(input, state); // Update @@ -322,7 +324,7 @@ }; // Add checked, disabled or indeterminate state - function on(input, state, keep) { + function on(input, state, keep) { var node = input[0], parent = input.parent(), checked = state == _checked, diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index 0f3c8b0e0..1a0bf2e32 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -65,20 +65,12 @@ module JamWebsockets @client_lookup[client_id] = client_context end - def remove_client(client_id, client) + def remove_client(client_id) deleted = @client_lookup.delete(client_id) if deleted.nil? - @log.warn "unable to delete #{client_id} from client_lookup" - elsif deleted.client != client - # put it back--this is only possible if add_client hit the 'old connection' path - # so in other words if this happens: - # add_client(1, clientX) - # add_client(1, clientY) # but clientX is essentially defunct - this could happen due to a bug in client, or EM doesn't notify always of connection close in time - # remove_client(1, clientX) -- this check maintains that clientY stays as the current client in the hash - @client_lookup[client_id] = deleted - @log.debug "putting back client into @client_lookup for #{client_id} #{client.inspect}" - else + @log.warn "unable to delete #{client_id} from client_lookup because it's already gone" + else @log.debug "cleaned up @client_lookup for #{client_id}" end @@ -377,7 +369,7 @@ module JamWebsockets @log.debug "cleanup up logged-in client #{client}" - remove_client(client.client_id, client) + remove_client(client.client_id) context = @clients.delete(client) @@ -462,6 +454,13 @@ module JamWebsockets user = valid_login(username, password, token, client_id) + # kill any websocket connections that have this same client_id, which can happen in race conditions + existing_client = @client_lookup[client_id] + if existing_client + remove_client(client_id) + existing_client.client.close_websocket + end + connection = JamRuby::Connection.find_by_client_id(client_id) # if this connection is reused by a different user, then whack the connection # because it will recreate a new connection lower down From 264d65d98bf657ab4c908c043a235a5f801b2b12 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Mon, 28 Apr 2014 19:51:27 +0000 Subject: [PATCH 09/20] * merged in more websocket fixes --- web/app/assets/javascripts/JamServer.js | 86 ++++++++++++++----------- 1 file changed, 49 insertions(+), 37 deletions(-) diff --git a/web/app/assets/javascripts/JamServer.js b/web/app/assets/javascripts/JamServer.js index aaf2cfdd6..98ed67c24 100644 --- a/web/app/assets/javascripts/JamServer.js +++ b/web/app/assets/javascripts/JamServer.js @@ -60,11 +60,11 @@ freezeInteraction = activeElementVotes && ((activeElementVotes.dialog && activeElementVotes.dialog.freezeInteraction === true) || (activeElementVotes.screen && activeElementVotes.screen.freezeInteraction === true)); - if(!initialConnect) { + if (!initialConnect) { context.JK.CurrentSessionModel.onWebsocketDisconnected(in_error); } - if(in_error) { + if (in_error) { reconnectAttempt = 0; $currentDisplay = renderDisconnected(); beginReconnectPeriod(); @@ -88,7 +88,7 @@ if (server.connected) { server.connected = false; - if(app.clientUpdating) { + if (app.clientUpdating) { // we don't want to do a 'cover the whole screen' dialog // because the client update is already showing. return; @@ -144,9 +144,9 @@ // for debugging purposes, see if the last time we've sent a heartbeat is way off (500ms) of the target interval var now = new Date(); - if(lastHeartbeatSentTime) { + if (lastHeartbeatSentTime) { var drift = new Date().getTime() - lastHeartbeatSentTime.getTime() - heartbeatMS; - if(drift > 500) { + if (drift > 500) { logger.error("significant drift between heartbeats: " + drift + 'ms beyond target interval') } } @@ -158,7 +158,7 @@ function loggedIn(header, payload) { - if(!connectTimeout) { + if (!connectTimeout) { clearTimeout(connectTimeout); connectTimeout = null; } @@ -220,10 +220,10 @@ function internetUp() { var start = new Date().getTime(); server.connect() - .done(function() { + .done(function () { guardAgainstRapidTransition(start, performReconnect); }) - .fail(function() { + .fail(function () { guardAgainstRapidTransition(start, closedOnReconnectAttempt); }); } @@ -235,14 +235,14 @@ function performReconnect() { - if($currentDisplay.is('.no-websocket-connection')) { + if ($currentDisplay.is('.no-websocket-connection')) { $currentDisplay.hide(); // TODO: tell certain elements that we've reconnected } else { context.JK.CurrentSessionModel.leaveCurrentSession() - .always(function() { + .always(function () { window.location.reload(); }); } @@ -256,14 +256,14 @@ function renderDisconnected() { var content = null; - if(freezeInteraction) { + if (freezeInteraction) { var template = $templateDisconnected.html(); var templateHtml = $(context.JK.fillTemplate(template, buildOptions())); templateHtml.find('.reconnect-countdown').html(formatDelaySecs(reconnectDelaySecs())); content = context.JK.Banner.show({ - html : templateHtml, + html: templateHtml, type: 'reconnect' - }) ; + }); } else { var $inSituContent = $(context._.template($templateServerConnection.html(), buildOptions(), { variable: 'data' })); @@ -278,7 +278,7 @@ } function formatDelaySecs(secs) { - return $('' + secs + ' ' + (secs == 1 ? ' second.s' : 'seconds.') + ''); + return $('' + secs + ' ' + (secs == 1 ? ' second.s' : 'seconds.') + ''); } function setCountdown($parent) { @@ -292,7 +292,7 @@ function renderReconnecting() { $currentDisplay.find('.reconnect-progress-msg').text('Attempting to reconnect...') - if($currentDisplay.is('.no-websocket-connection')) { + if ($currentDisplay.is('.no-websocket-connection')) { $currentDisplay.find('.disconnected-reconnect').removeClass('reconnect-enabled').addClass('reconnect-disabled'); } else { @@ -310,7 +310,7 @@ var now = new Date().getTime(); if ((now - start) < 1500) { - setTimeout(function() { + setTimeout(function () { nextStep(); }, 1500 - (now - start)) } @@ -326,12 +326,12 @@ renderReconnecting(); rest.serverHealthCheck() - .done(function() { + .done(function () { guardAgainstRapidTransition(start, internetUp); }) - .fail(function(xhr, textStatus, errorThrown) { + .fail(function (xhr, textStatus, errorThrown) { - if(xhr && xhr.status >= 100) { + if (xhr && xhr.status >= 100) { // we could connect to the server, and it's alive guardAgainstRapidTransition(start, internetUp); } @@ -344,7 +344,7 @@ } function clearReconnectTimers() { - if(countdownInterval) { + if (countdownInterval) { clearInterval(countdownInterval); countdownInterval = null; } @@ -352,8 +352,8 @@ function beginReconnectPeriod() { // allow user to force reconnect - $currentDisplay.find('a.disconnected-reconnect').unbind('click').click(function() { - if($(this).is('.button-orange') || $(this).is('.reconnect-enabled')) { + $currentDisplay.find('a.disconnected-reconnect').unbind('click').click(function () { + if ($(this).is('.button-orange') || $(this).is('.reconnect-enabled')) { clearReconnectTimers(); attemptReconnect(); } @@ -364,9 +364,9 @@ reconnectDueTime = reconnectingWaitPeriodStart + reconnectDelaySecs() * 1000; // update count down timer periodically - countdownInterval = setInterval(function() { + countdownInterval = setInterval(function () { var now = new Date().getTime(); - if(now > reconnectDueTime) { + if (now > reconnectDueTime) { clearReconnectTimers(); attemptReconnect(); } @@ -425,9 +425,9 @@ server.socket.onmessage = server.onMessage; server.socket.onclose = server.onClose; - connectTimeout = setTimeout(function() { + connectTimeout = setTimeout(function () { connectTimeout = null; - if(connectDeferred.state() === 'pending') { + if (connectDeferred.state() === 'pending') { server.close(true); connectDeferred.reject(); } @@ -486,7 +486,7 @@ server.onClose = function () { logger.log("Socket to server closed."); - if(connectDeferred.state() === "pending") { + if (connectDeferred.state() === "pending") { connectDeferred.reject(); } @@ -533,19 +533,19 @@ //console.timeEnd('sendP2PMessage'); }; - server.updateNotificationSeen = function(notificationId, notificationCreatedAt) { + server.updateNotificationSeen = function (notificationId, notificationCreatedAt) { var time = new Date(notificationCreatedAt); - if(!notificationCreatedAt) { + if (!notificationCreatedAt) { throw 'invalid value passed to updateNotificationSeen' } - if(!notificationLastSeenAt) { + if (!notificationLastSeenAt) { notificationLastSeenAt = notificationCreatedAt; notificationLastSeen = notificationId; logger.debug("updated notificationLastSeenAt with: " + notificationCreatedAt); } - else if(time.getTime() > new Date(notificationLastSeenAt).getTime()) { + else if (time.getTime() > new Date(notificationLastSeenAt).getTime()) { notificationLastSeenAt = notificationCreatedAt; notificationLastSeen = notificationId; logger.debug("updated notificationLastSeenAt with: " + notificationCreatedAt); @@ -596,12 +596,24 @@ $templateServerConnection = $('#template-server-connection'); $templateDisconnected = $('#template-disconnected'); - if($inSituBanner.length != 1) { throw "found wrong number of .server-connection: " + $inSituBanner.length; } - if($inSituBannerHolder.length != 1) { throw "found wrong number of .no-websocket-connection: " + $inSituBannerHolder.length; } - if($messageContents.length != 1) { throw "found wrong number of .message-contents: " + $messageContents.length; } - if($dialog.length != 1) { throw "found wrong number of #banner: " + $dialog.length; } - if($templateServerConnection.length != 1) { throw "found wrong number of #template-server-connection: " + $templateServerConnection.length; } - if($templateDisconnected.length != 1) { throw "found wrong number of #template-disconnected: " + $templateDisconnected.length; } + if ($inSituBanner.length != 1) { + throw "found wrong number of .server-connection: " + $inSituBanner.length; + } + if ($inSituBannerHolder.length != 1) { + throw "found wrong number of .no-websocket-connection: " + $inSituBannerHolder.length; + } + if ($messageContents.length != 1) { + throw "found wrong number of .message-contents: " + $messageContents.length; + } + if ($dialog.length != 1) { + throw "found wrong number of #banner: " + $dialog.length; + } + if ($templateServerConnection.length != 1) { + throw "found wrong number of #template-server-connection: " + $templateServerConnection.length; + } + if ($templateDisconnected.length != 1) { + throw "found wrong number of #template-disconnected: " + $templateDisconnected.length; + } } this.initialize = initialize; From ece5ee80f86d175780ce4e483a01b1e341c2e133 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Tue, 29 Apr 2014 01:45:06 +0000 Subject: [PATCH 10/20] * debugging websocket-gateway problems --- .../lib/jam_websockets/router.rb | 68 +++++++++++++------ .../lib/jam_websockets/server.rb | 17 ++++- 2 files changed, 63 insertions(+), 22 deletions(-) diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index 1a0bf2e32..dc1c20f66 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -11,6 +11,22 @@ module EventMachine module WebSocket class Connection < EventMachine::Connection attr_accessor :encode_json, :client_id # client_id is uuid we give to each client to track them as we like + + # http://stackoverflow.com/questions/11150147/how-to-check-if-eventmachineconnection-is-open + attr_accessor :connected + def connection_completed + connected = true + super + end + + def connected? + !!connected + end + + def unbind + connected = false + super + end end end end @@ -35,7 +51,7 @@ module JamWebsockets @client_topic = nil @thread_pool = nil @heartbeat_interval = nil - + @ar_base_logger = ::Logging::Repository.instance[ActiveRecord::Base] end def start(connect_time_stale, options={:host => "localhost", :port => 5672}, &block) @@ -230,12 +246,15 @@ module JamWebsockets @log.error "generic error: #{error} #{error.backtrace}" end - cleanup_client(client) - client.close_websocket + unless error.to_s.include? "Close handshake un-acked" + cleanup_client(client) + else + @log.info "skipping cleanup because error is for dead connection: https://github.com/igrigorik/em-websocket/issues/122" + end + } client.onmessage { |msg| - @log.debug("msg received") # TODO: set a max message size before we put it through PB? # TODO: rate limit? @@ -259,7 +278,7 @@ module JamWebsockets error_msg = @message_factory.server_rejection_error(e.to_s) send_to_client(client, error_msg) ensure - client.close_websocket + client.close cleanup_client(client) end rescue PermissionError => e @@ -278,7 +297,7 @@ module JamWebsockets error_msg = @message_factory.server_generic_error(e.to_s) send_to_client(client, error_msg) ensure - client.close_websocket + client.close cleanup_client(client) end end @@ -287,7 +306,7 @@ module JamWebsockets def send_to_client(client, msg) - @log.debug "SEND TO CLIENT (#{@message_factory.get_message_type(msg)})" + @log.debug "SEND TO CLIENT (#{@message_factory.get_message_type(msg)})" unless msg.type == ClientMessage::Type::HEARTBEAT_ACK if client.encode_json client.send(msg.to_json.to_s) else @@ -360,25 +379,23 @@ module JamWebsockets # removes all resources associated with a client def cleanup_client(client) @semaphore.synchronize do + client.close if client.connected? + # @log.debug("*** cleanup_clients: client = #{client}") pending = @pending_clients.delete?(client) - if !pending.nil? - @log.debug "cleaning up not-logged-in client #{client}" + if pending + @log.debug "cleaned up not-logged-in client #{client}" else - @log.debug "cleanup up logged-in client #{client}" - - remove_client(client.client_id) - context = @clients.delete(client) - if !context.nil? + if context + remove_client(client.client_id) remove_user(context) else @log.debug "skipping duplicate cleanup attempt of logged-in client" end - end end end @@ -388,7 +405,7 @@ module JamWebsockets raise SessionError, "unknown message type received: #{client_msg.type}" if message_type.nil? - @log.debug("msg received #{message_type}") + @log.debug("msg received #{message_type}") if client_msg.type != ClientMessage::Type::HEARTBEAT raise SessionError, 'client_msg.route_to is null' if client_msg.route_to.nil? @@ -426,9 +443,7 @@ module JamWebsockets handle_login(client_msg.login, client) elsif client_msg.type == ClientMessage::Type::HEARTBEAT - - handle_heartbeat(client_msg.heartbeat, client_msg.message_id, client) - + sane_logging { handle_heartbeat(client_msg.heartbeat, client_msg.message_id, client) } else raise SessionError, "unknown message type '#{client_msg.type}' for #{client_msg.route_to}-directed message" end @@ -458,7 +473,7 @@ module JamWebsockets existing_client = @client_lookup[client_id] if existing_client remove_client(client_id) - existing_client.client.close_websocket + existing_client.client.close end connection = JamRuby::Connection.find_by_client_id(client_id) @@ -767,5 +782,18 @@ module JamWebsockets def extract_ip(client) return Socket.unpack_sockaddr_in(client.get_peername)[1] end + + private + + def sane_logging(&blk) + # used around repeated transactions that cause too much ActiveRecord::Base logging + begin + original_level = @ar_base_logger.level + @ar_base_logger.level = :info if @ar_base_logger + blk.call + ensure + @ar_base_logger.level = original_level if @ar_base_logger + end + end end end diff --git a/websocket-gateway/lib/jam_websockets/server.rb b/websocket-gateway/lib/jam_websockets/server.rb index 4bf384396..1277936cb 100644 --- a/websocket-gateway/lib/jam_websockets/server.rb +++ b/websocket-gateway/lib/jam_websockets/server.rb @@ -6,9 +6,11 @@ module JamWebsockets class Server def initialize(options={}) + EM::WebSocket.close_timeout = 10 # the default of 60 is pretty intense @log = Logging.logger[self] @count=0 @router = Router.new + @ar_base_logger = ::Logging::Repository.instance[ActiveRecord::Base] end def run(options={}) @@ -64,7 +66,7 @@ module JamWebsockets expire_stale_connections(stale_max_time) EventMachine::PeriodicTimer.new(stale_max_time) do - expire_stale_connections(stale_max_time) + sane_logging { expire_stale_connections(stale_max_time) } end end @@ -83,7 +85,7 @@ module JamWebsockets flag_stale_connections(flag_max_time) EventMachine::PeriodicTimer.new(flag_max_time/2) do - flag_stale_connections(flag_max_time) + sane_logging { flag_stale_connections(flag_max_time) } end end @@ -94,6 +96,17 @@ module JamWebsockets end end + def sane_logging(&blk) + # used around repeated transactions that cause too much ActiveRecord::Base logging + begin + original_level = @ar_base_logger.level + @ar_base_logger.level = :info if @ar_base_logger + blk.call + ensure + @ar_base_logger.level = original_level if @ar_base_logger + end + end + end end From 3e2f39cc85e1a8e6a6cd644f7a95da52d8f07cfe Mon Sep 17 00:00:00 2001 From: Seth Call Date: Tue, 29 Apr 2014 16:27:50 +0000 Subject: [PATCH 11/20] * VRFS-1653 - don't bother with cleanup in onerror, and make startup of EventMachine better (VRFS-1659) --- ruby/lib/jam_ruby/lib/em_helper.rb | 15 ++++++++++++--- websocket-gateway/lib/jam_websockets/router.rb | 7 ------- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/ruby/lib/jam_ruby/lib/em_helper.rb b/ruby/lib/jam_ruby/lib/em_helper.rb index 14d661c06..2d3c4a77c 100644 --- a/ruby/lib/jam_ruby/lib/em_helper.rb +++ b/ruby/lib/jam_ruby/lib/em_helper.rb @@ -36,8 +36,9 @@ module JamWebEventMachine end - def self.run_em(calling_thread = nil) + def self.run_em(calling_thread = nil, semaphore = nil, ran = {}) + semaphore.lock if semaphore EM.run do # this is global because we need to check elsewhere if we are currently connected to amqp before signalling success with some APIs, such as 'create session' $amqp_connection_manager = AmqpConnectionManager.new(true, 4, :host => APP_CONFIG.rabbitmq_host, :port => APP_CONFIG.rabbitmq_port) @@ -54,6 +55,8 @@ module JamWebEventMachine end end + ran[:ran] = true + semaphore.unlock if semaphore calling_thread.wakeup if calling_thread end end @@ -66,11 +69,17 @@ module JamWebEventMachine def self.run + ran = {} + semaphore = Mutex.new current = Thread.current Thread.new do - run_em(current) + run_em(current, semaphore, ran) end - Thread.stop + semaphore.synchronize { + unless ran[:ran] + semaphore.sleep(10) + end + } end def self.start diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index dc1c20f66..de56c7774 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -245,13 +245,6 @@ module JamWebsockets else @log.error "generic error: #{error} #{error.backtrace}" end - - unless error.to_s.include? "Close handshake un-acked" - cleanup_client(client) - else - @log.info "skipping cleanup because error is for dead connection: https://github.com/igrigorik/em-websocket/issues/122" - end - } client.onmessage { |msg| From d472c2b78fde1223384ad38b19c635245d39da40 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Tue, 29 Apr 2014 16:29:06 +0000 Subject: [PATCH 12/20] * keeping master manifest is lock with develop --- db/manifest | 3 +++ 1 file changed, 3 insertions(+) diff --git a/db/manifest b/db/manifest index 232cb12ef..61b2dba30 100755 --- a/db/manifest +++ b/db/manifest @@ -143,3 +143,6 @@ emails.sql email_batch.sql user_progress_tracking2.sql bands_did_session.sql +email_change_default_sender.sql +affiliate_partners.sql +chat_messages.sql From e3e9cb58307e8f9644be9b7ea269004a6d329eba Mon Sep 17 00:00:00 2001 From: Seth Call Date: Tue, 29 Apr 2014 16:31:36 +0000 Subject: [PATCH 13/20] * keeping master in line with development, for dataabse --- db/up/affiliate_partners.sql | 15 +++++++++++++++ db/up/chat_messages.sql | 8 ++++++++ db/up/email_change_default_sender.sql | 1 + 3 files changed, 24 insertions(+) create mode 100644 db/up/affiliate_partners.sql create mode 100644 db/up/chat_messages.sql create mode 100644 db/up/email_change_default_sender.sql diff --git a/db/up/affiliate_partners.sql b/db/up/affiliate_partners.sql new file mode 100644 index 000000000..67f7dfd97 --- /dev/null +++ b/db/up/affiliate_partners.sql @@ -0,0 +1,15 @@ +CREATE TABLE affiliate_partners ( + id VARCHAR(64) PRIMARY KEY DEFAULT uuid_generate_v4(), + partner_name VARCHAR(128) NOT NULL, + partner_code VARCHAR(128) NOT NULL, + partner_user_id VARCHAR(64) NOT NULL, + user_email VARCHAR(255), + referral_user_count INTEGER NOT NULL DEFAULT 0, + created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +CREATE INDEX affiliate_partners_code_idx ON affiliate_partners(partner_code); +CREATE INDEX affiliate_partners_user_idx ON affiliate_partners(partner_user_id); + +ALTER TABLE users ADD COLUMN affiliate_referral_id VARCHAR(64) REFERENCES affiliate_partners(id); diff --git a/db/up/chat_messages.sql b/db/up/chat_messages.sql new file mode 100644 index 000000000..01a0fcd14 --- /dev/null +++ b/db/up/chat_messages.sql @@ -0,0 +1,8 @@ +CREATE TABLE chat_messages +( + id character varying(64) NOT NULL DEFAULT uuid_generate_v4(), + user_id character varying(64), + music_session_id character varying(64), + messsage TEXT NOT NULL, + created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP +); \ No newline at end of file diff --git a/db/up/email_change_default_sender.sql b/db/up/email_change_default_sender.sql new file mode 100644 index 000000000..e729980f0 --- /dev/null +++ b/db/up/email_change_default_sender.sql @@ -0,0 +1 @@ +ALTER TABLE email_batches ALTER column from_email SET DEFAULT 'noreply@jamkazam.com'::character varying; \ No newline at end of file From 982056289265d8d24a7bfe59247af99e0ceed47a Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 30 Apr 2014 03:01:28 +0000 Subject: [PATCH 14/20] * more websocket/heartbeat fixes --- db/manifest | 3 + db/up/connection_stale_expire.sql | 2 + db/up/diagnostics.sql | 11 ++ db/up/user_mods.sql | 1 + pb/src/client_container.proto | 1 + ruby/lib/jam_ruby.rb | 2 + ruby/lib/jam_ruby/connection_manager.rb | 13 ++- ruby/lib/jam_ruby/lib/json_validator.rb | 15 +++ ruby/lib/jam_ruby/message_factory.rb | 5 +- ruby/lib/jam_ruby/models/diagnostic.rb | 85 ++++++++++++++ ruby/lib/jam_ruby/models/email_batch.rb | 2 +- ruby/lib/jam_ruby/models/user.rb | 21 +++- ruby/spec/factories.rb | 6 + ruby/spec/jam_ruby/connection_manager_spec.rb | 6 +- ruby/spec/jam_ruby/models/diagnostic_spec.rb | 18 +++ ruby/spec/jam_ruby/models/user_spec.rb | 42 +++++++ web/app/assets/javascripts/AAA_Log.js | 52 ++++++--- web/app/assets/javascripts/JamServer.js | 30 ++++- web/app/assets/javascripts/jam_rest.js | 11 ++ web/app/assets/javascripts/utils.js | 3 + .../controllers/api_diagnostics_controller.rb | 16 +++ web/config/initializers/eventmachine.rb | 2 +- web/config/routes.rb | 3 + web/spec/requests/diagnostics_api_spec.rb | 43 +++++++ web/spec/spec_helper.rb | 2 +- .../assets/javascripts/jquery.icheck.js | 2 - websocket-gateway/bin/websocket_gateway | 6 +- websocket-gateway/config/application.yml | 6 +- .../lib/jam_websockets/client_context.rb | 11 +- .../lib/jam_websockets/router.rb | 107 ++++++++++-------- .../lib/jam_websockets/server.rb | 24 ++-- .../jam_websockets/client_context_spec.rb | 2 +- .../spec/jam_websockets/router_spec.rb | 4 +- 33 files changed, 454 insertions(+), 103 deletions(-) create mode 100644 db/up/connection_stale_expire.sql create mode 100644 db/up/diagnostics.sql create mode 100644 db/up/user_mods.sql create mode 100644 ruby/lib/jam_ruby/lib/json_validator.rb create mode 100644 ruby/lib/jam_ruby/models/diagnostic.rb create mode 100644 ruby/spec/jam_ruby/models/diagnostic_spec.rb create mode 100644 web/app/controllers/api_diagnostics_controller.rb create mode 100644 web/spec/requests/diagnostics_api_spec.rb diff --git a/db/manifest b/db/manifest index 61b2dba30..e333987b2 100755 --- a/db/manifest +++ b/db/manifest @@ -146,3 +146,6 @@ bands_did_session.sql email_change_default_sender.sql affiliate_partners.sql chat_messages.sql +diagnostics.sql +user_mods.sql +connection_stale_expire.sql \ No newline at end of file diff --git a/db/up/connection_stale_expire.sql b/db/up/connection_stale_expire.sql new file mode 100644 index 000000000..57f34a1f1 --- /dev/null +++ b/db/up/connection_stale_expire.sql @@ -0,0 +1,2 @@ +ALTER TABLE connections ADD COLUMN stale_time INTEGER NOT NULL DEFAULT 20; +ALTER TABLE connections ADD COLUMN expire_time INTEGER NOT NULL DEFAULT 30; \ No newline at end of file diff --git a/db/up/diagnostics.sql b/db/up/diagnostics.sql new file mode 100644 index 000000000..5bb29399d --- /dev/null +++ b/db/up/diagnostics.sql @@ -0,0 +1,11 @@ +CREATE TABLE diagnostics +( + id VARCHAR(64) NOT NULL DEFAULT uuid_generate_v4(), + user_id VARCHAR(64) NOT NULL REFERENCES users (id) ON DELETE CASCADE, + type VARCHAR(255) NOT NULL, + creator VARCHAR(255) NOT NULL, + data TEXT, + created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP +); + +CREATE INDEX diagnostics_type_idx ON diagnostics(type); \ No newline at end of file diff --git a/db/up/user_mods.sql b/db/up/user_mods.sql new file mode 100644 index 000000000..41e9d1940 --- /dev/null +++ b/db/up/user_mods.sql @@ -0,0 +1 @@ +ALTER TABLE users ADD COLUMN mods JSON; \ No newline at end of file diff --git a/pb/src/client_container.proto b/pb/src/client_container.proto index 1df3bebf8..93bae50b5 100644 --- a/pb/src/client_container.proto +++ b/pb/src/client_container.proto @@ -184,6 +184,7 @@ message LoginAck { optional string music_session_id = 5; // the music session that the user was in very recently (likely due to dropped connection) optional bool reconnected = 6; // if reconnect_music_session_id is specified, and the server could log the user into that session, then true is returned. optional string user_id = 7; // the database user id + optional int32 connection_expire_time = 8; // this is how long the server gives you before killing your connection entirely after missing heartbeats } // route_to: server diff --git a/ruby/lib/jam_ruby.rb b/ruby/lib/jam_ruby.rb index ef426e9a8..bb158356a 100755 --- a/ruby/lib/jam_ruby.rb +++ b/ruby/lib/jam_ruby.rb @@ -31,6 +31,7 @@ require "jam_ruby/lib/module_overrides" require "jam_ruby/lib/s3_util" require "jam_ruby/lib/s3_manager" require "jam_ruby/lib/profanity" +require "jam_ruby/lib/json_validator" require "jam_ruby/lib/em_helper.rb" require "jam_ruby/lib/nav.rb" require "jam_ruby/resque/audiomixer" @@ -75,6 +76,7 @@ require "jam_ruby/models/artifact_update" require "jam_ruby/models/band_invitation" require "jam_ruby/models/band_musician" require "jam_ruby/models/connection" +require "jam_ruby/models/diagnostic" require "jam_ruby/models/friendship" require "jam_ruby/models/music_session" require "jam_ruby/models/music_session_comment" diff --git a/ruby/lib/jam_ruby/connection_manager.rb b/ruby/lib/jam_ruby/connection_manager.rb index faa505096..dd96b6765 100644 --- a/ruby/lib/jam_ruby/connection_manager.rb +++ b/ruby/lib/jam_ruby/connection_manager.rb @@ -129,7 +129,6 @@ WHERE aasm_state = '#{Connection::CONNECT_STATE.to_s}' RETURNING music_session_id SQL - # @log.info("*** flag_connection_stale_with_client_id: client_id = #{client_id}; sql = #{sql}") self.pg_conn.exec(sql) do |result| # if we did update a client to stale, retriee music_session_id @@ -171,16 +170,16 @@ SQL # NOTE this is only used for testing purposes; # actual deletes will be processed in the websocket context which cleans up dependencies def expire_stale_connections(max_seconds) - self.stale_connection_client_ids(max_seconds).each { |cid| self.delete_connection(cid) } + self.stale_connection_client_ids(max_seconds).each { |client| self.delete_connection(client[:client_id]) } end # expiring connections in stale state, which deletes them def stale_connection_client_ids(max_seconds) - client_ids = [] + clients = [] ConnectionManager.active_record_transaction do |connection_manager| conn = connection_manager.pg_conn sql =< public_ip, :client_id => client_id, @@ -61,7 +61,8 @@ module JamRuby :heartbeat_interval => heartbeat_interval, :music_session_id => music_session_id, :reconnected => reconnected, - :user_id => user_id + :user_id => user_id, + :connection_expire_time => connection_expire_time ) Jampb::ClientMessage.new( diff --git a/ruby/lib/jam_ruby/models/diagnostic.rb b/ruby/lib/jam_ruby/models/diagnostic.rb new file mode 100644 index 000000000..2315e0b9b --- /dev/null +++ b/ruby/lib/jam_ruby/models/diagnostic.rb @@ -0,0 +1,85 @@ +module JamRuby + class Diagnostic < ActiveRecord::Base + + # occurs when the client does not see a heartbeat from the server in a while + NO_HEARTBEAT_ACK = 'NO_HEARTBEAT_ACK' + + # occurs when the client sees the socket go down + WEBSOCKET_CLOSED_REMOTELY = 'WEBSOCKET_CLOSED_REMOTELY' + + # occurs when the websocket-gateway has finally given up entirely on a connection with no heartbeats seen in a while + EXPIRED_STALE_CONNECTION = 'EXPIRED_STALE_CONNECTION' + + # occurs when the websocket-gateway is trying to handle a heartbeat, but can't find any state for the user. + # this implies a coding error + MISSING_CLIENT_STATE = 'MISSING_CLIENT_STATE' + + # websocket gateway did not recognize message. indicates out-of-date websocket-gateway + UNKNOWN_MESSAGE_TYPE = 'UNKNOWN_MESSAGE_TYPE' + + # empty route_to in message; which is invalid. indicates programming error + MISSING_ROUTE_TO = 'MISSING_ROUTE_TO' + + # websocket gateway got a client with the same client_id as an already-connected client + DUPLICATE_CLIENT = 'DUPLICATE_CLIENT' + + DIAGNOSTIC_TYPES = [NO_HEARTBEAT_ACK, WEBSOCKET_CLOSED_REMOTELY, EXPIRED_STALE_CONNECTION, + MISSING_CLIENT_STATE, UNKNOWN_MESSAGE_TYPE, MISSING_ROUTE_TO, + DUPLICATE_CLIENT] + + # creator types # + CLIENT = 'client' + WEBSOCKET_GATEWAY = 'websocket-gateway' + CREATORS = [CLIENT, WEBSOCKET_GATEWAY] + + self.primary_key = 'id' + + belongs_to :user, :inverse_of => :diagnostics, :class_name => "JamRuby::User", :foreign_key => "user_id" + + validates :user, :presence => true + validates :type, :inclusion => {:in => DIAGNOSTIC_TYPES} + validates :creator, :inclusion => {:in => CREATORS} + validates :data, length: {maximum: 100000} + + + def self.expired_stale_connection(user, context_as_json) + Diagnostic.save(EXPIRED_STALE_CONNECTION, user, WEBSOCKET_GATEWAY, context_as_json) if user + end + + def self.missing_client_state(user, context) + Diagnostic.save(MISSING_CLIENT_STATE, user, WEBSOCKET_GATEWAY, context.to_json) if user + end + + def self.missing_connection(user, context) + Diagnostic.save(MISSING_CONNECTION, user, WEBSOCKET_GATEWAY, context.to_json) if user + end + + def self.duplicate_client(user, context) + Diagnostic.save(DUPLICATE_CLIENT, user, WEBSOCKET_GATEWAY, context.to_json) if user + end + + def self.unknown_message_type(user, client_msg) + Diagnostic.save(UNKNOWN_MESSAGE_TYPE, user, WEBSOCKET_GATEWAY, client_msg.to_json) if user + end + + def self.missing_route_to(user, client_msg) + Diagnostic.save(MISSING_ROUTE_TO, user, WEBSOCKET_GATEWAY, client_msg.to_json) if user + end + + + def self.save(type, user, creator, data) + diagnostic = Diagnostic.new + if user.class == String + diagnostic.user_id = user + else + diagnostic.user = user + end + + diagnostic.data = data + diagnostic.type = type + diagnostic.creator = creator + diagnostic.save + end + end + +end diff --git a/ruby/lib/jam_ruby/models/email_batch.rb b/ruby/lib/jam_ruby/models/email_batch.rb index 309b63305..74dff06ec 100644 --- a/ruby/lib/jam_ruby/models/email_batch.rb +++ b/ruby/lib/jam_ruby/models/email_batch.rb @@ -12,7 +12,7 @@ module JamRuby VAR_FIRST_NAME = '@FIRSTNAME' VAR_LAST_NAME = '@LASTNAME' - DEFAULT_SENDER = "support@jamkazam.com" + DEFAULT_SENDER = "noreply@jamkazam.com" BATCH_SIZE = 1000 BODY_TEMPLATE =< "JamRuby::EventSession" + # diagnostics + has_many :diagnostics, :class_name => "JamRuby::Diagnostic" + # This causes the authenticate method to be generated (among other stuff) #has_secure_password @@ -119,6 +122,7 @@ module JamRuby validates :subscribe_email, :inclusion => {:in => [nil, true, false]} validates :musician, :inclusion => {:in => [true, false]} validates :show_whats_next, :inclusion => {:in => [nil, true, false]} + validates :mods, json: true # custom validators validate :validate_musician_instruments @@ -280,6 +284,19 @@ module JamRuby self.music_sessions.size end + # mods comes back as text; so give ourselves a parsed version + def mods_json + @mods_json ||= mods ? JSON.parse(mods, symbolize_names: true) : {} + end + + def heartbeat_interval + mods_json[:heartbeat_interval] + end + + def connection_expire_time + mods_json[:connection_expire_time] + end + def recent_history recordings = Recording.where(:owner_id => self.id) .order('created_at DESC') @@ -356,7 +373,7 @@ module JamRuby return first_name + ' ' + last_name end - return id + id end def set_password(old_password, new_password, new_password_confirmation) diff --git a/ruby/spec/factories.rb b/ruby/spec/factories.rb index 2b5b85d88..47f1de842 100644 --- a/ruby/spec/factories.rb +++ b/ruby/spec/factories.rb @@ -443,4 +443,10 @@ FactoryGirl.define do message Faker::Lorem.characters(10) end end + + factory :diagnostic, :class => JamRuby::Diagnostic do + type JamRuby::Diagnostic::NO_HEARTBEAT_ACK + creator JamRuby::Diagnostic::CLIENT + data Faker::Lorem.sentence + end end diff --git a/ruby/spec/jam_ruby/connection_manager_spec.rb b/ruby/spec/jam_ruby/connection_manager_spec.rb index ee492fa41..853951416 100644 --- a/ruby/spec/jam_ruby/connection_manager_spec.rb +++ b/ruby/spec/jam_ruby/connection_manager_spec.rb @@ -261,7 +261,11 @@ describe ConnectionManager do cids = @connman.stale_connection_client_ids(1) cids.size.should == 1 - cids[0].should == client_id + cids[0][:client_id].should == client_id + cids[0][:client_type].should == 'native' + cids[0][:music_session_id].should be_nil + cids[0][:user_id].should == user_id + cids.each { |cid| @connman.delete_connection(cid) } sleep(1) diff --git a/ruby/spec/jam_ruby/models/diagnostic_spec.rb b/ruby/spec/jam_ruby/models/diagnostic_spec.rb new file mode 100644 index 000000000..8900deb21 --- /dev/null +++ b/ruby/spec/jam_ruby/models/diagnostic_spec.rb @@ -0,0 +1,18 @@ +require 'spec_helper' + +describe Diagnostic do + let (:user) { FactoryGirl.create(:user) } + let (:diagnostic) { FactoryGirl.create(:diagnostic, user: user) } + + it 'can be made' do + diagnostic.save! + end + + it "validates type" do + diagnostic = FactoryGirl.build(:diagnostic, user: user, type: 'bleh') + + diagnostic.errors[:type].should == [] + + end + +end diff --git a/ruby/spec/jam_ruby/models/user_spec.rb b/ruby/spec/jam_ruby/models/user_spec.rb index 2a0e3ebfa..956b7cff1 100644 --- a/ruby/spec/jam_ruby/models/user_spec.rb +++ b/ruby/spec/jam_ruby/models/user_spec.rb @@ -21,6 +21,7 @@ describe User do it { should respond_to(:admin) } it { should respond_to(:valid_password?) } it { should respond_to(:can_invite) } + it { should respond_to(:mods) } it { should be_valid } it { should_not be_admin } @@ -69,6 +70,24 @@ describe User do it { should_not be_valid } end + describe "when mods is null" do + before { @user.mods = nil } + it { should be_valid } + end + + describe "when mods is empty" do + before { @user.mods = 'nil' } + it { should_not be_valid } + end + + + describe "when mods is json object" do + before { @user.mods = '{"key":"value"}' } + it { should be_valid } + end + + + describe "first or last name cant have profanity" do it "should not let the first name have profanity" do @user.first_name = "fuck you" @@ -429,6 +448,29 @@ describe User do end + + describe "mods" do + it "should allow update of JSON" do + @user.mods = {some_field: 5}.to_json + @user.save! + end + + it "should return heartbeart interval" do + @user.heartbeat_interval_client.should be_nil + @user.mods = {heartbeat_interval_client: 5}.to_json + @user.save! + @user = User.find(@user.id) # necessary because mods_json is cached in the model + @user.heartbeat_interval_client.should == 5 + end + + it "should return connection_expire_time" do + @user.connection_expire_time.should be_nil + @user.mods = {connection_expire_time: 5}.to_json + @user.save! + @user = User.find(@user.id) # necessary because mods_json is cached in the model + @user.connection_expire_time.should == 5 + end + end =begin describe "update avatar" do diff --git a/web/app/assets/javascripts/AAA_Log.js b/web/app/assets/javascripts/AAA_Log.js index cd16f22b6..bdd411e38 100644 --- a/web/app/assets/javascripts/AAA_Log.js +++ b/web/app/assets/javascripts/AAA_Log.js @@ -15,6 +15,12 @@ 'exception', 'table' ]; + var log_methods = { + 'log':null, 'debug':null, 'info':null, 'warn':null, 'error':null, 'assert':null, 'trace':null, 'exception':null + } + + var logCache = []; + if ('undefined' === typeof(context.console)) { context.console = {}; $.each(console_methods, function(index, value) { @@ -27,23 +33,39 @@ context.console.debug = function() { console.log(arguments); } } - context.JK.logger = context.console; + // http://tobyho.com/2012/07/27/taking-over-console-log/ + function takeOverConsole(){ + var console = window.console + if (!console) return + function intercept(method){ + var original = console[method] + console[method] = function(){ - // JW - some code to tone down logging. Uncomment the following, and - // then do your logging to logger.dbg - and it will be the only thing output. - // TODO - find a way to wrap this up so that debug logs can stay in, but this - // class can provide a way to enable/disable certain namespaces of logs. - /* - var fakeLogger = {}; - $.each(console_methods, function(index, value) { - fakeLogger[value] = $.noop; - }); - fakeLogger.dbg = function(m) { - context.console.debug(m); - }; - context.JK.logger = fakeLogger; - */ + logCache.push([method].concat(arguments)); + if(logCache.length > 50) { + // keep the cache size 50 or lower + logCache.pop(); + } + if (original.apply){ + // Do this for normal browsers + original.apply(console, arguments) + }else{ + // Do this for IE + var message = Array.prototype.slice.apply(arguments).join(' ') + original(message) + } + } + } + var methods = ['log', 'warn', 'error'] + for (var i = 0; i < methods.length; i++) + intercept(methods[i]) + } + + takeOverConsole(); + + context.JK.logger = context.console; + context.JK.logger.logCache = logCache; })(window, jQuery); \ No newline at end of file diff --git a/web/app/assets/javascripts/JamServer.js b/web/app/assets/javascripts/JamServer.js index 98ed67c24..de7fde2e4 100644 --- a/web/app/assets/javascripts/JamServer.js +++ b/web/app/assets/javascripts/JamServer.js @@ -21,9 +21,11 @@ var lastHeartbeatSentTime = null; var lastHeartbeatAckTime = null; var lastHeartbeatFound = false; + var lastDisconnectedReason = null; var heartbeatAckCheckInterval = null; var notificationLastSeenAt = undefined; var notificationLastSeen = undefined; + var clientClosedConnection = false; // reconnection logic var connectDeferred = null; @@ -53,6 +55,13 @@ server.socketClosedListeners = []; server.connected = false; + var clientType = context.JK.clientType(); + + function heartbeatStateReset() { + lastHeartbeatSentTime = null; + lastHeartbeatAckTime = null; + lastHeartbeatFound = false; + } // if activeElementVotes is null, then we are assuming this is the initial connect sequence function initiateReconnect(activeElementVotes, in_error) { @@ -129,6 +138,7 @@ // this logic equates to 'if we have not received a heartbeat within heartbeatMissedMS, then get upset if (new Date().getTime() - lastHeartbeatAckTime.getTime() > heartbeatMissedMS) { logger.error("no heartbeat ack received from server after ", heartbeatMissedMS, " seconds . giving up on socket connection"); + lastDisconnectedReason = 'NO_HEARTBEAT_ACK'; context.JK.JamServer.close(true); } else { @@ -163,6 +173,8 @@ connectTimeout = null; } + heartbeatStateReset(); + app.clientId = payload.client_id; // tell the backend that we have logged in @@ -170,6 +182,7 @@ $.cookie('client_id', payload.client_id); + heartbeatMS = payload.heartbeat_interval * 1000; logger.debug("jamkazam.js.loggedIn(): clientId now " + app.clientId + "; Setting up heartbeat every " + heartbeatMS + " MS"); heartbeatInterval = context.setInterval(_heartbeat, heartbeatMS); @@ -235,12 +248,19 @@ function performReconnect() { + rest.createDiagnostic({ + type: lastDisconnectedReason, + data: {logs: logger.logCache, client_type: clientType, client_id: server.clientID} + }); + if ($currentDisplay.is('.no-websocket-connection')) { + // this path is the 'not in session path'; so there is nothing else to do $currentDisplay.hide(); // TODO: tell certain elements that we've reconnected } else { + // this path is the 'in session' path, where we actually reload the page context.JK.CurrentSessionModel.leaveCurrentSession() .always(function () { window.location.reload(); @@ -439,6 +459,7 @@ server.close = function (in_error) { logger.log("closing websocket"); + clientClosedConnection = true; server.socket.close(); closedCleanup(in_error); @@ -447,7 +468,7 @@ server.rememberLogin = function () { var token, loginMessage; token = $.cookie("remember_token"); - var clientType = context.jamClient.IsNativeClient() ? 'client' : 'browser'; + loginMessage = msg_factory.login_with_token(token, null, clientType); server.send(loginMessage); }; @@ -483,13 +504,18 @@ } }; + // onClose is called if either client or server closes connection server.onClose = function () { - logger.log("Socket to server closed."); + logger.log("Socket to server closed.", arguments); if (connectDeferred.state() === "pending") { connectDeferred.reject(); } + if(!clientClosedConnection) { + lastDisconnectedReason = 'WEBSOCKET_CLOSED_REMOTELY' + clientClosedConnection = false; + } closedCleanup(true); }; diff --git a/web/app/assets/javascripts/jam_rest.js b/web/app/assets/javascripts/jam_rest.js index fbdf74a2a..c2cd8a89e 100644 --- a/web/app/assets/javascripts/jam_rest.js +++ b/web/app/assets/javascripts/jam_rest.js @@ -944,6 +944,16 @@ }); } + function createDiagnostic(options) { + return $.ajax({ + type: "POST", + url: '/api/diagnostics', + dataType: "json", + contentType: 'application/json', + data: JSON.stringify(options) + }); + } + function initialize() { return self; } @@ -1026,6 +1036,7 @@ this.createFbInviteUrl = createFbInviteUrl; this.createTextMessage = createTextMessage; this.getNotifications = getNotifications; + this.createDiagnostic = createDiagnostic; return this; }; diff --git a/web/app/assets/javascripts/utils.js b/web/app/assets/javascripts/utils.js index 9890d9c43..5cf9af4ea 100644 --- a/web/app/assets/javascripts/utils.js +++ b/web/app/assets/javascripts/utils.js @@ -572,6 +572,9 @@ doneYet(); }; + context.JK.clientType = function () { + return context.jamClient.IsNativeClient() ? 'client' : 'browser'; + } /** * Returns 'MacOSX' if the os appears to be macintosh, * 'Win32' if the os appears to be windows, diff --git a/web/app/controllers/api_diagnostics_controller.rb b/web/app/controllers/api_diagnostics_controller.rb new file mode 100644 index 000000000..ac8a17ce9 --- /dev/null +++ b/web/app/controllers/api_diagnostics_controller.rb @@ -0,0 +1,16 @@ +class ApiDiagnosticsController < ApiController + + before_filter :api_signed_in_user + respond_to :json + + def create + @diagnostic = Diagnostic.new + @diagnostic.type = params[:type] + @diagnostic.data = params[:data].to_json if params[:data] + @diagnostic.user = current_user + @diagnostic.creator = Diagnostic::CLIENT + @diagnostic.save + + respond_with_model(@diagnostic, new: true) + end +end diff --git a/web/config/initializers/eventmachine.rb b/web/config/initializers/eventmachine.rb index 547cd5ec8..64cf993cc 100644 --- a/web/config/initializers/eventmachine.rb +++ b/web/config/initializers/eventmachine.rb @@ -10,7 +10,7 @@ unless $rails_rake_task :port => APP_CONFIG.websocket_gateway_port, :emwebsocket_debug => APP_CONFIG.websocket_gateway_internal_debug, :connect_time_stale => APP_CONFIG.websocket_gateway_connect_time_stale, - :connect_time_expire => APP_CONFIG.websocket_gateway_connect_time_expire, + :connect_time_expire_client => APP_CONFIG.websocket_gateway_connect_time_expire, :rabbitmq_host => APP_CONFIG.rabbitmq_host, :rabbitmq_port => APP_CONFIG.rabbitmq_port, :calling_thread => current) diff --git a/web/config/routes.rb b/web/config/routes.rb index 02153f193..e8e34ee9f 100644 --- a/web/config/routes.rb +++ b/web/config/routes.rb @@ -402,6 +402,9 @@ SampleApp::Application.routes.draw do # favorites match '/favorites' => 'api_favorites#index', :via => :get match '/favorites/:id' => 'api_favorites#update', :via => :post + + # diagnostic + match '/diagnostics' => 'api_diagnostics#create', :via => :post end end diff --git a/web/spec/requests/diagnostics_api_spec.rb b/web/spec/requests/diagnostics_api_spec.rb new file mode 100644 index 000000000..ca677866d --- /dev/null +++ b/web/spec/requests/diagnostics_api_spec.rb @@ -0,0 +1,43 @@ +require 'spec_helper' + +# user progression is achieved by different aspects of the code working together in a cross-cutting fashion. +# due to this, it's nice to have a single place where all the parts of user progression are tested +# https://jamkazam.atlassian.net/wiki/pages/viewpage.action?pageId=3375145 + +describe "Diagnostics", :type => :api do + + include Rack::Test::Methods + + let(:user) { FactoryGirl.create(:user) } + + subject { page } + + def login(user) + post '/sessions', "session[email]" => user.email, "session[password]" => user.password + rack_mock_session.cookie_jar["remember_token"].should == user.remember_token + end + + + describe "create" do + + before do + Diagnostic.delete_all + login(user) + end + + it "can fail" do + post "/api/diagnostics.json", {}.to_json, "CONTENT_TYPE" => 'application/json' + last_response.status.should eql(422) + JSON.parse(last_response.body).should eql({"errors"=>{"type"=>["is not included in the list"], "creator"=>["is not included in the list"]}}) + Diagnostic.count.should == 0 + end + + it "can succeed" do + post "/api/diagnostics.json", { type: Diagnostic::NO_HEARTBEAT_ACK}.to_json, "CONTENT_TYPE" => 'application/json' + last_response.status.should eql(201) + Diagnostic.count.should == 1 + end + + + end +end diff --git a/web/spec/spec_helper.rb b/web/spec/spec_helper.rb index d3f3dccbe..598a19c3e 100644 --- a/web/spec/spec_helper.rb +++ b/web/spec/spec_helper.rb @@ -76,7 +76,7 @@ Thread.new do :port => 6769, :emwebsocket_debug => false, :connect_time_stale => 2, - :connect_time_expire => 5, + :connect_time_expire_client => 5, :rabbitmq_host => 'localhost', :rabbitmq_port => 5672, :calling_thread => current) diff --git a/web/vendor/assets/javascripts/jquery.icheck.js b/web/vendor/assets/javascripts/jquery.icheck.js index 4e0cffeb9..ab4eed092 100644 --- a/web/vendor/assets/javascripts/jquery.icheck.js +++ b/web/vendor/assets/javascripts/jquery.icheck.js @@ -285,12 +285,10 @@ // Check, disable or indeterminate if (/^(ch|di|in)/.test(method) && !active) { - console.log("TAKING ROUTE: ", state); on(input, state); // Uncheck, enable or determinate } else if (/^(un|en|de)/.test(method) && active) { - console.log("TAKING ROUTE2: ", state); off(input, state); // Update diff --git a/websocket-gateway/bin/websocket_gateway b/websocket-gateway/bin/websocket_gateway index 343f589dc..e4a544209 100755 --- a/websocket-gateway/bin/websocket_gateway +++ b/websocket-gateway/bin/websocket_gateway @@ -47,7 +47,9 @@ Object.send(:remove_const, :Rails) # this is to 'fool' new relic into not thinki Server.new.run(:port => config["port"], :emwebsocket_debug => config["emwebsocket_debug"], - :connect_time_stale => config["connect_time_stale"], - :connect_time_expire => config["connect_time_expire"], + :connect_time_stale_client => config["connect_time_stale_client"], + :connect_time_expire_client => config["connect_time_expire_client"], + :connect_time_stale_browser => config["connect_time_stale_browser"], + :connect_time_expire_browser => config["connect_time_expire_browser"], :rabbitmq_host => config['rabbitmq_host'], :rabbitmq_port => config['rabbitmq_port']) diff --git a/websocket-gateway/config/application.yml b/websocket-gateway/config/application.yml index 7647b2df8..b25f37cea 100644 --- a/websocket-gateway/config/application.yml +++ b/websocket-gateway/config/application.yml @@ -1,6 +1,8 @@ Defaults: &defaults - connect_time_stale: 6 - connect_time_expire: 10 + connect_time_stale_client: 20 + connect_time_expire_client: 30 + connect_time_stale_browser: 40 + connect_time_expire_browser: 60 development: port: 6767 diff --git a/websocket-gateway/lib/jam_websockets/client_context.rb b/websocket-gateway/lib/jam_websockets/client_context.rb index c3f302778..ecb065118 100644 --- a/websocket-gateway/lib/jam_websockets/client_context.rb +++ b/websocket-gateway/lib/jam_websockets/client_context.rb @@ -1,20 +1,27 @@ module JamWebsockets class ClientContext - attr_accessor :user, :client, :msg_count, :session, :sent_bad_state_previously + attr_accessor :user, :client, :msg_count, :session, :client_type, :sent_bad_state_previously - def initialize(user, client) + def initialize(user, client, client_type) @user = user @client = client + + @client_type = client_type @msg_count = 0 @session = nil @sent_bad_state_previously = false + client.context = self end def to_s return "Client[user:#{@user} client:#{@client} msgs:#{@msg_count} session:#{@session}]" end + def to_json + {user_id: @user.id, client_id: @client.client_id, msg_count: @msg_count, client_type: @client_type}.to_json + end + def hash @client.hash end diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index de56c7774..5ccd62ab7 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -10,7 +10,7 @@ include Jampb module EventMachine module WebSocket class Connection < EventMachine::Connection - attr_accessor :encode_json, :client_id # client_id is uuid we give to each client to track them as we like + attr_accessor :encode_json, :client_id, :user_id, :context # client_id is uuid we give to each client to track them as we like # http://stackoverflow.com/questions/11150147/how-to-check-if-eventmachineconnection-is-open attr_accessor :connected @@ -39,7 +39,6 @@ module JamWebsockets def initialize() @log = Logging.logger[self] - @pending_clients = Set.new # clients that have connected to server, but not logged in. @clients = {} # clients that have logged in @user_context_lookup = {} # lookup a set of client_contexts by user_id @client_lookup = {} # lookup a client by client_id @@ -50,15 +49,19 @@ module JamWebsockets @user_topic = nil @client_topic = nil @thread_pool = nil - @heartbeat_interval = nil + @heartbeat_interval_client = nil + @connect_time_expire_client = nil + @heartbeat_interval_browser= nil + @connect_time_expire_browser= nil @ar_base_logger = ::Logging::Repository.instance[ActiveRecord::Base] end - def start(connect_time_stale, options={:host => "localhost", :port => 5672}, &block) + def start(connect_time_stale_client, connect_time_expire_client, connect_time_stale_browser, connect_time_expire_browser, options={:host => "localhost", :port => 5672}, &block) @log.info "startup" - @heartbeat_interval = connect_time_stale / 2 + @heartbeat_interval_client = connect_time_stale_client / 2 + @connect_time_expire_client = connect_time_expire_client begin @amqp_connection_manager = AmqpConnectionManager.new(true, 4, :host => options[:host], :port => options[:port]) @@ -214,10 +217,6 @@ module JamWebsockets def new_client(client) - @semaphore.synchronize do - @pending_clients.add(client) - end - # default to using json instead of pb client.encode_json = true @@ -237,6 +236,7 @@ module JamWebsockets client.onclose { @log.debug "Connection closed" stale_client(client) + cleanup_client(client) } client.onerror { |error| @@ -271,7 +271,6 @@ module JamWebsockets error_msg = @message_factory.server_rejection_error(e.to_s) send_to_client(client, error_msg) ensure - client.close cleanup_client(client) end rescue PermissionError => e @@ -290,7 +289,6 @@ module JamWebsockets error_msg = @message_factory.server_generic_error(e.to_s) send_to_client(client, error_msg) ensure - client.close cleanup_client(client) end end @@ -328,9 +326,9 @@ module JamWebsockets # caused a client connection to be marked stale def stale_client(client) - if cid = client.client_id + if client.client_id ConnectionManager.active_record_transaction do |connection_manager| - music_session_id = connection_manager.flag_connection_stale_with_client_id(cid) + music_session_id = connection_manager.flag_connection_stale_with_client_id(client.client_id) # update the session members, letting them know this client went stale context = @client_lookup[client.client_id] if music_session = MusicSession.find_by_id(music_session_id) @@ -340,12 +338,14 @@ module JamWebsockets end end - def cleanup_clients_with_ids(client_ids) - # @log.debug("*** cleanup_clients_with_ids: client_ids = #{client_ids.inspect}") - client_ids.each do |cid| + def cleanup_clients_with_ids(expired_connections) + expired_connections.each do |expired_connection| + cid = expired_connection[:client_id] client_context = @client_lookup[cid] - self.cleanup_client(client_context.client) unless client_context.nil? + + diagnostic_data = client_context.to_json unless client_context.nil? + cleanup_client(client_context.client) unless client_context.nil? music_session = nil recordingId = nil @@ -354,6 +354,7 @@ module JamWebsockets # remove this connection from the database ConnectionManager.active_record_transaction do |mgr| mgr.delete_connection(cid) { |conn, count, music_session_id, user_id| + Diagnostic.expired_stale_connection(user_id, diagnostic_data) Notification.send_friend_update(user_id, false, conn) if count == 0 music_session = MusicSession.find_by_id(music_session_id) unless music_session_id.nil? user = User.find_by_id(user_id) unless user_id.nil? @@ -374,8 +375,7 @@ module JamWebsockets @semaphore.synchronize do client.close if client.connected? - # @log.debug("*** cleanup_clients: client = #{client}") - pending = @pending_clients.delete?(client) + pending = client.context.nil? # presence of context implies this connection has been logged into if pending @log.debug "cleaned up not-logged-in client #{client}" @@ -387,7 +387,7 @@ module JamWebsockets remove_client(client.client_id) remove_user(context) else - @log.debug "skipping duplicate cleanup attempt of logged-in client" + @log.warn "skipping duplicate cleanup attempt of logged-in client" end end end @@ -395,14 +395,19 @@ module JamWebsockets def route(client_msg, client) message_type = @message_factory.get_message_type(client_msg) - - raise SessionError, "unknown message type received: #{client_msg.type}" if message_type.nil? + if message_type.nil? + Diagnostic.unknown_message_type(client.user_id, client_msg) + raise SessionError, "unknown message type received: #{client_msg.type}" if message_type.nil? + end @log.debug("msg received #{message_type}") if client_msg.type != ClientMessage::Type::HEARTBEAT - raise SessionError, 'client_msg.route_to is null' if client_msg.route_to.nil? + if client_msg.route_to.nil? + Diagnostic.missing_route_to(client.user_id, client_msg) + raise SessionError, 'client_msg.route_to is null' + end - if @pending_clients.include? client and client_msg.type != ClientMessage::Type::LOGIN + if !client.user_id and client_msg.type != ClientMessage::Type::LOGIN # this client has not logged in and is trying to send a non-login message raise SessionError, "must 'Login' first" end @@ -463,25 +468,28 @@ module JamWebsockets user = valid_login(username, password, token, client_id) # kill any websocket connections that have this same client_id, which can happen in race conditions - existing_client = @client_lookup[client_id] - if existing_client - remove_client(client_id) - existing_client.client.close + # this code must happen here, before we go any further, so that there is only one websocket connection per client_id + existing_context = @client_lookup[client_id] + if existing_context + # in reconnect scenarios, we may have in memory a client still + Diagnostic.duplicate_client(existing_context.user, existing_context) if existing_context.client.connected + cleanup_client(existing_context.client) end connection = JamRuby::Connection.find_by_client_id(client_id) # if this connection is reused by a different user, then whack the connection # because it will recreate a new connection lower down - if !connection.nil? && !user.nil? && connection.user != user + if connection && user && connection.user != user @log.debug("user #{user.email} took client_id #{client_id} from user #{connection.user.email}") connection.delete connection = nil end client.client_id = client_id + client.user_id = user.id if user remote_ip = extract_ip(client) - if !user.nil? + if user @log.debug "user #{user} logged in with client_id #{client_id}" # check if there's a connection for the client... if it's stale, reconnect it @@ -492,19 +500,17 @@ module JamWebsockets music_session_upon_reentry = connection.music_session send_depart = false - recordingId = nil - context = nil + recording_id = nil ConnectionManager.active_record_transaction do |connection_manager| music_session_id, reconnected = connection_manager.reconnect(connection, reconnect_music_session_id, remote_ip) - context = @client_lookup[client_id] if music_session_id.nil? # if this is a reclaim of a connection, but music_session_id comes back null, then we need to check if this connection was IN a music session before. # if so, then we need to tell the others in the session that this user is now departed - unless context.nil? || music_session_upon_reentry.nil? || music_session_upon_reentry.destroyed? + unless music_session_upon_reentry.nil? || music_session_upon_reentry.destroyed? recording = music_session_upon_reentry.stop_recording - recordingId = recording.id unless recording.nil? + recording_id = recording.id unless recording.nil? music_session_upon_reentry.with_lock do # VRFS-1297 music_session_upon_reentry.tick_track_changes end @@ -512,24 +518,21 @@ module JamWebsockets end else music_session = MusicSession.find_by_id(music_session_id) - Notification.send_musician_session_fresh(music_session, client.client_id, context.user) unless context.nil? + Notification.send_musician_session_fresh(music_session, client.client_id, user) end - end if connection.stale? + end if send_depart - Notification.send_session_depart(music_session_upon_reentry, client.client_id, context.user, recordingId) + Notification.send_session_depart(music_session_upon_reentry, client.client_id, user, recording_id) end end # respond with LOGIN_ACK to let client know it was successful - @semaphore.synchronize do - # remove from pending_queue - @pending_clients.delete(client) # add a tracker for this user - context = ClientContext.new(user, client) + context = ClientContext.new(user, client, client_type) @clients[client] = context add_user(context) add_client(client_id, context) @@ -544,13 +547,21 @@ module JamWebsockets end end end + + heartbeat_interval = user.heartbeat_interval_client.to_i || @heartbeat_interval_client + heartbeat_interval = @heartbeat_interval_client if heartbeat_interval == 0 # protect against bad config + connection_expire_time = user.connection_expire_time || @connection_expire_time + connection_expire_time = @connection_expire_time if connection_expire_time == 0 # protect against bad config + + login_ack = @message_factory.login_ack(remote_ip, client_id, user.remember_token, - @heartbeat_interval, + @heartbeat_interval_client, connection.try(:music_session_id), reconnected, - user.id) + user.id, + @connection_expire_time) send_to_client(client, login_ack) end else @@ -560,15 +571,15 @@ module JamWebsockets def handle_heartbeat(heartbeat, heartbeat_message_id, client) unless context = @clients[client] - @log.warn "*** WARNING: unable to find context due to heartbeat from client: #{client.client_id}; calling cleanup" - cleanup_client(client) + @log.warn "*** WARNING: unable to find context when handling heartbeat. client_id=#{client.client_id}; killing session" + Diagnostic.missing_client_state(client.user_id, client.context) raise SessionError, 'context state is gone. please reconnect.' else connection = Connection.find_by_user_id_and_client_id(context.user.id, context.client.client_id) track_changes_counter = nil if connection.nil? - @log.warn "*** WARNING: unable to find connection due to heartbeat from client: #{context}; calling cleanup_client" - cleanup_client(client) + @log.warn "*** WARNING: unable to find connection when handling heartbeat. context= #{context}; killing session" + Diagnostic.missing_connection(client.user_id, client.context) raise SessionError, 'connection state is gone. please reconnect.' else Connection.transaction do diff --git a/websocket-gateway/lib/jam_websockets/server.rb b/websocket-gateway/lib/jam_websockets/server.rb index 1277936cb..b9a8b9a92 100644 --- a/websocket-gateway/lib/jam_websockets/server.rb +++ b/websocket-gateway/lib/jam_websockets/server.rb @@ -16,13 +16,15 @@ module JamWebsockets def run(options={}) host = "0.0.0.0" port = options[:port] - connect_time_stale = options[:connect_time_stale].to_i - connect_time_expire = options[:connect_time_expire].to_i + connect_time_stale_client = options[:connect_time_stale_client].to_i + connect_time_expire_client = options[:connect_time_expire_client].to_i + connect_time_stale_browser = options[:connect_time_stale_browser].to_i + connect_time_expire_browser = options[:connect_time_expire_browser].to_i rabbitmq_host = options[:rabbitmq_host] rabbitmq_port = options[:rabbitmq_port].to_i calling_thread = options[:calling_thread] - @log.info "starting server #{host}:#{port} staleness_time=#{connect_time_stale}; reconnect time = #{connect_time_expire}, rabbitmq=#{rabbitmq_host}:#{rabbitmq_port}" + @log.info "starting server #{host}:#{port} staleness_time=#{connect_time_stale_client}; reconnect time = #{connect_time_expire_client}, rabbitmq=#{rabbitmq_host}:#{rabbitmq_port}" EventMachine.error_handler{|e| @log.error "unhandled error #{e}" @@ -30,13 +32,10 @@ module JamWebsockets } EventMachine.run do - @router.start(connect_time_stale, host: rabbitmq_host, port: rabbitmq_port) do - # take stale off the expire limit because the call to stale will - # touch the updated_at column, adding an extra stale limit to the expire time limit - # expire_time = connect_time_expire > connect_time_stale ? connect_time_expire - connect_time_stale : connect_time_expire - expire_time = connect_time_expire + @router.start(connect_time_stale_client, connect_time_expire_client, connect_time_stale_browser, connect_time_expire_browser, host: rabbitmq_host, port: rabbitmq_port) do + expire_time = connect_time_expire_client start_connection_expiration(expire_time) - start_connection_flagger(connect_time_stale) + start_connection_flagger(connect_time_stale_client) start_websocket_listener(host, port, options[:emwebsocket_debug]) calling_thread.wakeup if calling_thread end @@ -72,12 +71,11 @@ module JamWebsockets end def expire_stale_connections(stale_max_time) - client_ids = [] + clients = [] ConnectionManager.active_record_transaction do |connection_manager| - client_ids = connection_manager.stale_connection_client_ids(stale_max_time) + clients = connection_manager.stale_connection_client_ids(stale_max_time) end - # @log.debug("*** expire_stale_connections(#{stale_max_time}): client_ids = #{client_ids.inspect}") - @router.cleanup_clients_with_ids(client_ids) + @router.cleanup_clients_with_ids(clients) end def start_connection_flagger(flag_max_time) diff --git a/websocket-gateway/spec/jam_websockets/client_context_spec.rb b/websocket-gateway/spec/jam_websockets/client_context_spec.rb index 522e77b10..47f422ff4 100644 --- a/websocket-gateway/spec/jam_websockets/client_context_spec.rb +++ b/websocket-gateway/spec/jam_websockets/client_context_spec.rb @@ -2,7 +2,7 @@ require 'spec_helper' describe ClientContext do - let(:context) {ClientContext.new({}, "client1")} + let(:context) {ClientContext.new({}, "client1", "client")} describe 'hashing' do it "hash correctly" do diff --git a/websocket-gateway/spec/jam_websockets/router_spec.rb b/websocket-gateway/spec/jam_websockets/router_spec.rb index da423991d..e1df86433 100644 --- a/websocket-gateway/spec/jam_websockets/router_spec.rb +++ b/websocket-gateway/spec/jam_websockets/router_spec.rb @@ -42,7 +42,7 @@ def login(router, user, password, client_id) message_factory = MessageFactory.new client = LoginClient.new - login_ack = message_factory.login_ack("127.0.0.1", client_id, user.remember_token, 15, nil, false, user.id) + login_ack = message_factory.login_ack("127.0.0.1", client_id, user.remember_token, 15, nil, false, user.id, 30) router.should_receive(:send_to_client) do |*args| args.count.should == 2 @@ -126,7 +126,7 @@ describe Router do user = double(User) user.should_receive(:id).any_number_of_times.and_return("1") client = double("client") - context = ClientContext.new(user, client) + context = ClientContext.new(user, client, "client") @router.user_context_lookup.length.should == 0 From ad266e5b808e3e5faa7338fe4f50bc5953bf5d71 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Wed, 30 Apr 2014 15:29:10 -0500 Subject: [PATCH 15/20] * VRFS-1663 (diagnostics), VRFS-1657 (configurable timer for heartbeats), VRFS-1653 (websocket connection cleanup) --- db/up/connection_stale_expire.sql | 4 +- ruby/lib/jam_ruby/connection_manager.rb | 28 ++++--- ruby/lib/jam_ruby/models/connection.rb | 6 +- ruby/lib/jam_ruby/models/feed.rb | 2 +- ruby/lib/jam_ruby/models/user.rb | 8 +- .../jam_ruby/resque/icecast_config_writer.rb | 2 +- .../resque/scheduled/icecast_source_check.rb | 2 +- ruby/spec/jam_ruby/connection_manager_spec.rb | 6 +- ruby/spec/jam_ruby/models/user_spec.rb | 6 +- web/app/assets/javascripts/JamServer.js | 58 +++++++++------ web/app/views/clients/index.html.erb | 1 - web/config/application.rb | 12 ++- web/config/environments/development.rb | 6 +- web/config/initializers/eventmachine.rb | 6 +- web/spec/spec_helper.rb | 6 +- web/spec/support/utilities.rb | 4 +- websocket-gateway/config/application.yml | 6 +- .../lib/jam_websockets/client_context.rb | 2 +- .../lib/jam_websockets/router.rb | 74 ++++++++++++++----- .../lib/jam_websockets/server.rb | 29 ++++---- 20 files changed, 162 insertions(+), 106 deletions(-) diff --git a/db/up/connection_stale_expire.sql b/db/up/connection_stale_expire.sql index 57f34a1f1..a98953f1b 100644 --- a/db/up/connection_stale_expire.sql +++ b/db/up/connection_stale_expire.sql @@ -1,2 +1,2 @@ -ALTER TABLE connections ADD COLUMN stale_time INTEGER NOT NULL DEFAULT 20; -ALTER TABLE connections ADD COLUMN expire_time INTEGER NOT NULL DEFAULT 30; \ No newline at end of file +ALTER TABLE connections ADD COLUMN stale_time INTEGER NOT NULL DEFAULT 40; +ALTER TABLE connections ADD COLUMN expire_time INTEGER NOT NULL DEFAULT 60; \ No newline at end of file diff --git a/ruby/lib/jam_ruby/connection_manager.rb b/ruby/lib/jam_ruby/connection_manager.rb index dd96b6765..1f39755c6 100644 --- a/ruby/lib/jam_ruby/connection_manager.rb +++ b/ruby/lib/jam_ruby/connection_manager.rb @@ -44,7 +44,7 @@ module JamRuby end # reclaim the existing connection, if ip_address is not nil then perhaps a new address as well - def reconnect(conn, reconnect_music_session_id, ip_address) + def reconnect(conn, reconnect_music_session_id, ip_address, connection_stale_time, connection_expire_time) music_session_id = nil reconnected = false @@ -54,7 +54,7 @@ module JamRuby joined_session_at_expression = 'NULL' unless reconnect_music_session_id.nil? music_session_id_expression = "(CASE WHEN music_session_id='#{reconnect_music_session_id}' THEN music_session_id ELSE NULL END)" - joined_session_at_expression = "(CASE WHEN music_session_id='#{reconnect_music_session_id}' THEN NOW() ELSE NULL END)" + joined_session_at_expression = "(CASE WHEN music_session_id='#{reconnect_music_session_id}' THEN NOW() at time zone 'utc' ELSE NULL END)" end if ip_address and !ip_address.eql?(conn.ip_address) @@ -101,7 +101,7 @@ module JamRuby end sql =< "JamRuby::MusicSession" has_many :tracks, :class_name => "JamRuby::Track", :inverse_of => :connection, :foreign_key => 'connection_id', :dependent => :delete_all - validates :as_musician, :inclusion => {:in => [true, false]} - validates :client_type, :inclusion => {:in => ['client', 'browser']} + validates :client_type, :inclusion => {:in => [TYPE_CLIENT, TYPE_BROWSER]} validate :can_join_music_session, :if => :joining_session? after_save :require_at_least_one_track_when_in_session, :if => :joining_session? after_create :did_create diff --git a/ruby/lib/jam_ruby/models/feed.rb b/ruby/lib/jam_ruby/models/feed.rb index 64f002a1d..a04f9091f 100644 --- a/ruby/lib/jam_ruby/models/feed.rb +++ b/ruby/lib/jam_ruby/models/feed.rb @@ -61,7 +61,7 @@ module JamRuby # handle time range days = TIME_RANGES[time_range] if days > 0 - query = query.where("feeds.created_at > NOW() - '#{days} day'::INTERVAL") + query = query.where("feeds.created_at > NOW() at time zone 'utc' - '#{days} day'::INTERVAL") end # handle type filters diff --git a/ruby/lib/jam_ruby/models/user.rb b/ruby/lib/jam_ruby/models/user.rb index ee4ad0c60..44147da44 100644 --- a/ruby/lib/jam_ruby/models/user.rb +++ b/ruby/lib/jam_ruby/models/user.rb @@ -289,12 +289,12 @@ module JamRuby @mods_json ||= mods ? JSON.parse(mods, symbolize_names: true) : {} end - def heartbeat_interval - mods_json[:heartbeat_interval] + def heartbeat_interval_client + mods_json[:heartbeat_interval_client] end - def connection_expire_time - mods_json[:connection_expire_time] + def connection_expire_time_client + mods_json[:connection_expire_time_client] end def recent_history diff --git a/ruby/lib/jam_ruby/resque/icecast_config_writer.rb b/ruby/lib/jam_ruby/resque/icecast_config_writer.rb index 05ce40a26..c40be7d5a 100644 --- a/ruby/lib/jam_ruby/resque/icecast_config_writer.rb +++ b/ruby/lib/jam_ruby/resque/icecast_config_writer.rb @@ -21,7 +21,7 @@ module JamRuby def self.queue_jobs_needing_retry # if we haven't seen updated_at be tickled in 5 minutes, but config_changed is still set to TRUE, this record has gotten stale - IcecastServer.find_each(:conditions => "config_changed = 1 AND updated_at < (NOW() - interval '#{APP_CONFIG.icecast_max_missing_check} second')", :batch_size => 100) do |server| + IcecastServer.find_each(:conditions => "config_changed = 1 AND updated_at < (NOW() at time zone 'utc' - interval '#{APP_CONFIG.icecast_max_missing_check} second')", :batch_size => 100) do |server| IcecastConfigWriter.enqueue(server.server_id) end end diff --git a/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb b/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb index 4927e6568..14749829f 100644 --- a/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb +++ b/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb @@ -32,7 +32,7 @@ module JamRuby def run # if we haven't seen updated_at be tickled in 5 minutes, but config_changed is still set to TRUE, this record has gotten stale - IcecastMount.find_each(lock: true, :conditions => "sourced_needs_changing_at < (NOW() - interval '#{APP_CONFIG.icecast_max_sourced_changed} second')", :batch_size => 100) do |mount| + IcecastMount.find_each(lock: true, :conditions => "sourced_needs_changing_at < (NOW() at time zone 'utc' - interval '#{APP_CONFIG.icecast_max_sourced_changed} second')", :batch_size => 100) do |mount| if mount.music_session_id mount.with_lock do handle_notifications(mount) diff --git a/ruby/spec/jam_ruby/connection_manager_spec.rb b/ruby/spec/jam_ruby/connection_manager_spec.rb index 853951416..6ae0fc029 100644 --- a/ruby/spec/jam_ruby/connection_manager_spec.rb +++ b/ruby/spec/jam_ruby/connection_manager_spec.rb @@ -247,15 +247,15 @@ describe ConnectionManager do sleep(1) - num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() - interval '#{1} second') AND aasm_state = 'connected'"]) + num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() at time zone 'utc' - interval '#{1} second') AND aasm_state = 'connected'"]) num.should == 1 # this should change the aasm_state to stale @connman.flag_stale_connections(1) - num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() - interval '#{1} second') AND aasm_state = 'connected'"]) + num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() at time zone 'utc' - interval '#{1} second') AND aasm_state = 'connected'"]) num.should == 0 - num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() - interval '#{1} second') AND aasm_state = 'stale'"]) + num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() at time zone 'utc' - interval '#{1} second') AND aasm_state = 'stale'"]) num.should == 1 assert_num_connections(client_id, 1) diff --git a/ruby/spec/jam_ruby/models/user_spec.rb b/ruby/spec/jam_ruby/models/user_spec.rb index 956b7cff1..0e0396fa3 100644 --- a/ruby/spec/jam_ruby/models/user_spec.rb +++ b/ruby/spec/jam_ruby/models/user_spec.rb @@ -464,11 +464,11 @@ describe User do end it "should return connection_expire_time" do - @user.connection_expire_time.should be_nil - @user.mods = {connection_expire_time: 5}.to_json + @user.connection_expire_time_client.should be_nil + @user.mods = {connection_expire_time_client: 5}.to_json @user.save! @user = User.find(@user.id) # necessary because mods_json is cached in the model - @user.connection_expire_time.should == 5 + @user.connection_expire_time_client.should == 5 end end =begin diff --git a/web/app/assets/javascripts/JamServer.js b/web/app/assets/javascripts/JamServer.js index de7fde2e4..077df1392 100644 --- a/web/app/assets/javascripts/JamServer.js +++ b/web/app/assets/javascripts/JamServer.js @@ -14,10 +14,14 @@ context.JK.JamServer = function (app) { + // uniquely identify the websocket connection + var channelId = null; + var clientType = null; + // heartbeat var heartbeatInterval = null; var heartbeatMS = null; - var heartbeatMissedMS = 10000; // if 10 seconds go by and we haven't seen a heartbeat ack, get upset + var connection_expire_time = null; var lastHeartbeatSentTime = null; var lastHeartbeatAckTime = null; var lastHeartbeatFound = false; @@ -55,7 +59,6 @@ server.socketClosedListeners = []; server.connected = false; - var clientType = context.JK.clientType(); function heartbeatStateReset() { lastHeartbeatSentTime = null; @@ -136,8 +139,8 @@ // check if the server is still sending heartbeat acks back down // this logic equates to 'if we have not received a heartbeat within heartbeatMissedMS, then get upset - if (new Date().getTime() - lastHeartbeatAckTime.getTime() > heartbeatMissedMS) { - logger.error("no heartbeat ack received from server after ", heartbeatMissedMS, " seconds . giving up on socket connection"); + if (new Date().getTime() - lastHeartbeatAckTime.getTime() > connection_expire_time) { + logger.error("no heartbeat ack received from server after ", connection_expire_time, " seconds . giving up on socket connection"); lastDisconnectedReason = 'NO_HEARTBEAT_ACK'; context.JK.JamServer.close(true); } @@ -184,11 +187,11 @@ heartbeatMS = payload.heartbeat_interval * 1000; - logger.debug("jamkazam.js.loggedIn(): clientId now " + app.clientId + "; Setting up heartbeat every " + heartbeatMS + " MS"); + connection_expire_time = payload.connection_expire_time * 1000; + logger.debug("jamkazam.js.loggedIn(): clientId=" + app.clientId + ", heartbeat=" + heartbeatMS + "ms, expire_time=" + connection_expire_time); heartbeatInterval = context.setInterval(_heartbeat, heartbeatMS); heartbeatAckCheckInterval = context.setInterval(_heartbeatAckCheck, 1000); lastHeartbeatAckTime = new Date(new Date().getTime() + heartbeatMS); // add a little forgiveness to server for initial heartbeat - connectDeferred.resolve(); app.activeElementEvent('afterConnect', payload); @@ -250,23 +253,26 @@ rest.createDiagnostic({ type: lastDisconnectedReason, - data: {logs: logger.logCache, client_type: clientType, client_id: server.clientID} + data: {logs: logger.logCache, client_type: clientType, client_id: server.clientID, channel_id: channelId} + }) + .always(function() { + if ($currentDisplay.is('.no-websocket-connection')) { + // this path is the 'not in session path'; so there is nothing else to do + $currentDisplay.hide(); + + // TODO: tell certain elements that we've reconnected + } + else { + // this path is the 'in session' path, where we actually reload the page + context.JK.CurrentSessionModel.leaveCurrentSession() + .always(function () { + window.location.reload(); + }); + } + server.reconnecting = false; }); - if ($currentDisplay.is('.no-websocket-connection')) { - // this path is the 'not in session path'; so there is nothing else to do - $currentDisplay.hide(); - // TODO: tell certain elements that we've reconnected - } - else { - // this path is the 'in session' path, where we actually reload the page - context.JK.CurrentSessionModel.leaveCurrentSession() - .always(function () { - window.location.reload(); - }); - } - server.reconnecting = false; } function buildOptions() { @@ -435,9 +441,14 @@ }; server.connect = function () { + if(!clientType) { + clientType = context.JK.clientType(); + } connectDeferred = new $.Deferred(); - logger.log("server.connect"); - var uri = context.JK.websocket_gateway_uri; // Set in index.html.erb. + channelId = context.JK.generateUUID(); // create a new channel ID for every websocket connection + logger.log("connecting websocket, channel_id: " + channelId); + + var uri = context.JK.websocket_gateway_uri + '?channel_id=' + channelId; // Set in index.html.erb. //var uri = context.gon.websocket_gateway_uri; // Leaving here for now, as we're looking for a better solution. server.socket = new context.WebSocket(uri); @@ -506,7 +517,7 @@ // onClose is called if either client or server closes connection server.onClose = function () { - logger.log("Socket to server closed.", arguments); + logger.log("Socket to server closed."); if (connectDeferred.state() === "pending") { connectDeferred.reject(); @@ -611,6 +622,7 @@ } function initialize() { + registerLoginAck(); registerHeartbeatAck(); registerSocketClosed(); diff --git a/web/app/views/clients/index.html.erb b/web/app/views/clients/index.html.erb index bd37726f2..b9fbc1d71 100644 --- a/web/app/views/clients/index.html.erb +++ b/web/app/views/clients/index.html.erb @@ -298,7 +298,6 @@ window.jamClient = interceptedJamClient; - } // Let's get things rolling... diff --git a/web/config/application.rb b/web/config/application.rb index 5175e0e1d..aa2078e65 100644 --- a/web/config/application.rb +++ b/web/config/application.rb @@ -105,11 +105,15 @@ if defined?(Bundler) # Websocket-gateway embedded configs config.websocket_gateway_enable = false if Rails.env=='test' - config.websocket_gateway_connect_time_stale = 2 - config.websocket_gateway_connect_time_expire = 5 + config.websocket_gateway_connect_time_stale_client = 4 + config.websocket_gateway_connect_time_expire_client = 6 + config.websocket_gateway_connect_time_stale_browser = 4 + config.websocket_gateway_connect_time_expire_browser = 6 else - config.websocket_gateway_connect_time_stale = 12 # 12 matches production - config.websocket_gateway_connect_time_expire = 20 # 20 matches production + config.websocket_gateway_connect_time_stale_client = 40 # 40 matches production + config.websocket_gateway_connect_time_expire_client = 60 # 60 matches production + config.websocket_gateway_connect_time_stale_browser = 40 # 40 matches production + config.websocket_gateway_connect_time_expire_browser = 60 # 60 matches production end config.websocket_gateway_internal_debug = false config.websocket_gateway_port = 6767 + ENV['JAM_INSTANCE'].to_i diff --git a/web/config/environments/development.rb b/web/config/environments/development.rb index 017f717a1..797e9aeba 100644 --- a/web/config/environments/development.rb +++ b/web/config/environments/development.rb @@ -68,8 +68,10 @@ SampleApp::Application.configure do # it's nice to have even admin accounts (which all the default ones are) generate GA data for testing config.ga_suppress_admin = false - config.websocket_gateway_connect_time_stale = 12 - config.websocket_gateway_connect_time_expire = 20 + config.websocket_gateway_connect_time_stale_client = 40 # 40 matches production + config.websocket_gateway_connect_time_expire_client = 60 # 60 matches production + config.websocket_gateway_connect_time_stale_browser = 40 # 40 matches production + config.websocket_gateway_connect_time_expire_browser = 60 # 60 matches production config.audiomixer_path = ENV['AUDIOMIXER_PATH'] || audiomixer_workspace_path || "/var/lib/audiomixer/audiomixer/audiomixerapp" diff --git a/web/config/initializers/eventmachine.rb b/web/config/initializers/eventmachine.rb index 64cf993cc..53858b4d9 100644 --- a/web/config/initializers/eventmachine.rb +++ b/web/config/initializers/eventmachine.rb @@ -9,8 +9,10 @@ unless $rails_rake_task JamWebsockets::Server.new.run( :port => APP_CONFIG.websocket_gateway_port, :emwebsocket_debug => APP_CONFIG.websocket_gateway_internal_debug, - :connect_time_stale => APP_CONFIG.websocket_gateway_connect_time_stale, - :connect_time_expire_client => APP_CONFIG.websocket_gateway_connect_time_expire, + :connect_time_stale_client => APP_CONFIG.websocket_gateway_connect_time_stale_client, + :connect_time_expire_client => APP_CONFIG.websocket_gateway_connect_time_expire_client, + :connect_time_stale_browser => APP_CONFIG.websocket_gateway_connect_time_stale_browser, + :connect_time_expire_browser=> APP_CONFIG.websocket_gateway_connect_time_expire_browser, :rabbitmq_host => APP_CONFIG.rabbitmq_host, :rabbitmq_port => APP_CONFIG.rabbitmq_port, :calling_thread => current) diff --git a/web/spec/spec_helper.rb b/web/spec/spec_helper.rb index 598a19c3e..94960bf13 100644 --- a/web/spec/spec_helper.rb +++ b/web/spec/spec_helper.rb @@ -75,8 +75,10 @@ Thread.new do JamWebsockets::Server.new.run( :port => 6769, :emwebsocket_debug => false, - :connect_time_stale => 2, - :connect_time_expire_client => 5, + :connect_time_stale_client => 4, + :connect_time_expire_client => 6, + :connect_time_stale_browser => 4, + :connect_time_expire_browser => 6, :rabbitmq_host => 'localhost', :rabbitmq_port => 5672, :calling_thread => current) diff --git a/web/spec/support/utilities.rb b/web/spec/support/utilities.rb index 3040a535f..deba66215 100644 --- a/web/spec/support/utilities.rb +++ b/web/spec/support/utilities.rb @@ -131,8 +131,8 @@ end def leave_music_session_sleep_delay # add a buffer to ensure WSG has enough time to expire - sleep_dur = (Rails.application.config.websocket_gateway_connect_time_stale + - Rails.application.config.websocket_gateway_connect_time_expire) * 1.4 + sleep_dur = (Rails.application.config.websocket_gateway_connect_time_stale_browser + + Rails.application.config.websocket_gateway_connect_time_expire_browser) * 1.4 sleep sleep_dur end diff --git a/websocket-gateway/config/application.yml b/websocket-gateway/config/application.yml index b25f37cea..1b5e37f1b 100644 --- a/websocket-gateway/config/application.yml +++ b/websocket-gateway/config/application.yml @@ -1,8 +1,8 @@ Defaults: &defaults - connect_time_stale_client: 20 - connect_time_expire_client: 30 + connect_time_stale_client: 40 + connect_time_expire_client: 62 connect_time_stale_browser: 40 - connect_time_expire_browser: 60 + connect_time_expire_browser: 62 development: port: 6767 diff --git a/websocket-gateway/lib/jam_websockets/client_context.rb b/websocket-gateway/lib/jam_websockets/client_context.rb index ecb065118..2c8c027cb 100644 --- a/websocket-gateway/lib/jam_websockets/client_context.rb +++ b/websocket-gateway/lib/jam_websockets/client_context.rb @@ -19,7 +19,7 @@ end def to_json - {user_id: @user.id, client_id: @client.client_id, msg_count: @msg_count, client_type: @client_type}.to_json + {user_id: @user.id, client_id: @client.client_id, msg_count: @msg_count, client_type: @client_type, socket_id: @client.socket_id}.to_json end def hash diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index 5ccd62ab7..8ddb254c4 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -10,7 +10,7 @@ include Jampb module EventMachine module WebSocket class Connection < EventMachine::Connection - attr_accessor :encode_json, :client_id, :user_id, :context # client_id is uuid we give to each client to track them as we like + attr_accessor :encode_json, :channel_id, :client_id, :user_id, :context # client_id is uuid we give to each client to track them as we like # http://stackoverflow.com/questions/11150147/how-to-check-if-eventmachineconnection-is-open attr_accessor :connected @@ -51,8 +51,10 @@ module JamWebsockets @thread_pool = nil @heartbeat_interval_client = nil @connect_time_expire_client = nil + @connect_time_stale_client = nil @heartbeat_interval_browser= nil @connect_time_expire_browser= nil + @connect_time_stale_browser= nil @ar_base_logger = ::Logging::Repository.instance[ActiveRecord::Base] end @@ -60,8 +62,12 @@ module JamWebsockets @log.info "startup" - @heartbeat_interval_client = connect_time_stale_client / 2 - @connect_time_expire_client = connect_time_expire_client + @heartbeat_interval_client = connect_time_stale_client / 2 + @connect_time_stale_client = connect_time_stale_client + @connect_time_expire_client = connect_time_expire_client + @heartbeat_interval_browser = connect_time_stale_browser / 2 + @connect_time_stale_browser = connect_time_stale_browser + @connect_time_expire_browser = connect_time_expire_browser begin @amqp_connection_manager = AmqpConnectionManager.new(true, 4, :host => options[:host], :port => options[:port]) @@ -221,8 +227,11 @@ module JamWebsockets client.encode_json = true client.onopen { |handshake| - #binding.pry - @log.debug "client connected #{client}" + # a unique ID for this TCP connection, to aid in debugging + client.channel_id = handshake.query["channel_id"] + + @log.debug "client connected #{client} with channel_id: #{client.channel_id}" + # check for '?pb' or '?pb=true' in url query parameters query_pb = handshake.query["pb"] @@ -447,6 +456,37 @@ module JamWebsockets end end + # returns heartbeat_interval, connection stale time, and connection expire time + def determine_connection_times(user, client_type) + + if client_type == Connection::TYPE_BROWSER + default_heartbeat = @heartbeat_interval_browser + default_stale = @connect_time_stale_browser + default_expire = @connect_time_expire_browser + else + default_heartbeat = @heartbeat_interval_client + default_stale = @connect_time_stale_client + default_expire = @connect_time_expire_client + end + + heartbeat_interval = user.heartbeat_interval_client || default_heartbeat + heartbeat_interval = heartbeat_interval.to_i + heartbeat_interval = default_heartbeat if heartbeat_interval == 0 # protect against bad config + connection_expire_time = user.connection_expire_time_client || default_expire + connection_expire_time = connection_expire_time.to_i + connection_expire_time = default_expire if connection_expire_time == 0 # protect against bad config + connection_stale_time = default_stale # no user override exists for this; not a very meaningful time right now + + if heartbeat_interval >= connection_stale_time + raise SessionError, "misconfiguration! heartbeat_interval (#{heartbeat_interval}) should be less than stale time (#{connection_stale_time})" + end + if connection_stale_time >= connection_expire_time + raise SessionError, "misconfiguration! stale time (#{connection_stale_time}) should be less than expire time (#{connection_expire_time})" + end + + [heartbeat_interval, connection_stale_time, connection_expire_time] + end + def handle_login(login, client) username = login.username if login.value_for_tag(1) password = login.password if login.value_for_tag(2) @@ -460,8 +500,7 @@ module JamWebsockets # you don't have to supply client_id in login--if you don't, we'll generate one if client_id.nil? || client_id.empty? - # give a unique ID to this client. This is used to prevent session messages - # from echoing back to the sender, for instance. + # give a unique ID to this client. client_id = UUIDTools::UUID.random_create.to_s end @@ -471,13 +510,13 @@ module JamWebsockets # this code must happen here, before we go any further, so that there is only one websocket connection per client_id existing_context = @client_lookup[client_id] if existing_context - # in reconnect scenarios, we may have in memory a client still + # in some reconnect scenarios, we may have in memory a websocket client still. Diagnostic.duplicate_client(existing_context.user, existing_context) if existing_context.client.connected cleanup_client(existing_context.client) end connection = JamRuby::Connection.find_by_client_id(client_id) - # if this connection is reused by a different user, then whack the connection + # if this connection is reused by a different user (possible in logout/login scenarios), then whack the connection # because it will recreate a new connection lower down if connection && user && connection.user != user @log.debug("user #{user.email} took client_id #{client_id} from user #{connection.user.email}") @@ -490,6 +529,9 @@ module JamWebsockets remote_ip = extract_ip(client) if user + + heartbeat_interval, connection_stale_time, connection_expire_time = determine_connection_times(user, client_type) + @log.debug "user #{user} logged in with client_id #{client_id}" # check if there's a connection for the client... if it's stale, reconnect it @@ -503,7 +545,7 @@ module JamWebsockets recording_id = nil ConnectionManager.active_record_transaction do |connection_manager| - music_session_id, reconnected = connection_manager.reconnect(connection, reconnect_music_session_id, remote_ip) + music_session_id, reconnected = connection_manager.reconnect(connection, reconnect_music_session_id, remote_ip, connection_stale_time, connection_expire_time) if music_session_id.nil? # if this is a reclaim of a connection, but music_session_id comes back null, then we need to check if this connection was IN a music session before. @@ -540,7 +582,7 @@ module JamWebsockets unless connection # log this connection in the database ConnectionManager.active_record_transaction do |connection_manager| - connection_manager.create_connection(user.id, client.client_id, remote_ip, client_type) do |conn, count| + connection_manager.create_connection(user.id, client.client_id, remote_ip, client_type, connection_stale_time, connection_expire_time) do |conn, count| if count == 1 Notification.send_friend_update(user.id, true, conn) end @@ -548,20 +590,14 @@ module JamWebsockets end end - heartbeat_interval = user.heartbeat_interval_client.to_i || @heartbeat_interval_client - heartbeat_interval = @heartbeat_interval_client if heartbeat_interval == 0 # protect against bad config - connection_expire_time = user.connection_expire_time || @connection_expire_time - connection_expire_time = @connection_expire_time if connection_expire_time == 0 # protect against bad config - - login_ack = @message_factory.login_ack(remote_ip, client_id, user.remember_token, - @heartbeat_interval_client, + heartbeat_interval, connection.try(:music_session_id), reconnected, user.id, - @connection_expire_time) + connection_expire_time) send_to_client(client, login_ack) end else diff --git a/websocket-gateway/lib/jam_websockets/server.rb b/websocket-gateway/lib/jam_websockets/server.rb index b9a8b9a92..4aafa62c2 100644 --- a/websocket-gateway/lib/jam_websockets/server.rb +++ b/websocket-gateway/lib/jam_websockets/server.rb @@ -33,9 +33,8 @@ module JamWebsockets EventMachine.run do @router.start(connect_time_stale_client, connect_time_expire_client, connect_time_stale_browser, connect_time_expire_browser, host: rabbitmq_host, port: rabbitmq_port) do - expire_time = connect_time_expire_client - start_connection_expiration(expire_time) - start_connection_flagger(connect_time_stale_client) + start_connection_expiration() + start_connection_flagger() start_websocket_listener(host, port, options[:emwebsocket_debug]) calling_thread.wakeup if calling_thread end @@ -60,37 +59,37 @@ module JamWebsockets @log.debug("started websocket") end - def start_connection_expiration(stale_max_time) + def start_connection_expiration() # one cleanup on startup - expire_stale_connections(stale_max_time) + expire_stale_connections() - EventMachine::PeriodicTimer.new(stale_max_time) do - sane_logging { expire_stale_connections(stale_max_time) } + EventMachine::PeriodicTimer.new(5) do + sane_logging { expire_stale_connections() } end end - def expire_stale_connections(stale_max_time) + def expire_stale_connections() clients = [] ConnectionManager.active_record_transaction do |connection_manager| - clients = connection_manager.stale_connection_client_ids(stale_max_time) + clients = connection_manager.stale_connection_client_ids() end @router.cleanup_clients_with_ids(clients) end - def start_connection_flagger(flag_max_time) + def start_connection_flagger() # one cleanup on startup - flag_stale_connections(flag_max_time) + flag_stale_connections() - EventMachine::PeriodicTimer.new(flag_max_time/2) do - sane_logging { flag_stale_connections(flag_max_time) } + EventMachine::PeriodicTimer.new(5) do + sane_logging { flag_stale_connections() } end end - def flag_stale_connections(flag_max_time) + def flag_stale_connections() # @log.debug("*** flag_stale_connections: fires each #{flag_max_time} seconds") ConnectionManager.active_record_transaction do |connection_manager| - connection_manager.flag_stale_connections(flag_max_time) + connection_manager.flag_stale_connections() end end From c0ce26c6020d318a95933b9a1576897b3c0ee870 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Thu, 1 May 2014 14:09:33 -0500 Subject: [PATCH 16/20] * fixing tests for: * VRFS-1663 (diagnostics), VRFS-1657 (configurable timer for heartbeats), VRFS-1653 (websocket connection cleanup) --- ruby/lib/jam_ruby/connection_manager.rb | 14 ++- ruby/lib/jam_ruby/models/diagnostic.rb | 6 +- ruby/lib/jam_ruby/models/feed.rb | 2 +- ruby/lib/jam_ruby/models/friend_request.rb | 2 +- ruby/lib/jam_ruby/models/track.rb | 2 +- ruby/lib/jam_ruby/models/user.rb | 2 +- .../jam_ruby/resque/icecast_config_writer.rb | 2 +- .../resque/scheduled/icecast_source_check.rb | 2 +- ruby/spec/jam_ruby/connection_manager_spec.rb | 85 ++++++++++--------- ruby/spec/jam_ruby/models/track_spec.rb | 18 +--- .../resque/icecast_config_worker_spec.rb | 9 +- ruby/spec/support/utilities.rb | 12 +++ web/app/assets/javascripts/JamServer.js | 15 ++-- web/config/application.rb | 17 ++-- web/config/environments/test.rb | 5 ++ web/spec/features/reconnect_spec.rb | 8 +- web/spec/requests/diagnostics_api_spec.rb | 2 +- websocket-gateway/config/application.yml | 4 +- .../lib/jam_websockets/client_context.rb | 4 +- .../lib/jam_websockets/router.rb | 40 ++++++--- .../lib/jam_websockets/server.rb | 29 ++++--- .../jam_websockets/client_context_spec.rb | 8 +- .../spec/jam_websockets/router_spec.rb | 15 +++- 23 files changed, 173 insertions(+), 130 deletions(-) diff --git a/ruby/lib/jam_ruby/connection_manager.rb b/ruby/lib/jam_ruby/connection_manager.rb index 1f39755c6..d7846f8f1 100644 --- a/ruby/lib/jam_ruby/connection_manager.rb +++ b/ruby/lib/jam_ruby/connection_manager.rb @@ -54,7 +54,7 @@ module JamRuby joined_session_at_expression = 'NULL' unless reconnect_music_session_id.nil? music_session_id_expression = "(CASE WHEN music_session_id='#{reconnect_music_session_id}' THEN music_session_id ELSE NULL END)" - joined_session_at_expression = "(CASE WHEN music_session_id='#{reconnect_music_session_id}' THEN NOW() at time zone 'utc' ELSE NULL END)" + joined_session_at_expression = "(CASE WHEN music_session_id='#{reconnect_music_session_id}' THEN NOW() ELSE NULL END)" end if ip_address and !ip_address.eql?(conn.ip_address) @@ -101,7 +101,7 @@ module JamRuby end sql =< :diagnostics, :class_name => "JamRuby::User", :foreign_key => "user_id" diff --git a/ruby/lib/jam_ruby/models/feed.rb b/ruby/lib/jam_ruby/models/feed.rb index a04f9091f..64f002a1d 100644 --- a/ruby/lib/jam_ruby/models/feed.rb +++ b/ruby/lib/jam_ruby/models/feed.rb @@ -61,7 +61,7 @@ module JamRuby # handle time range days = TIME_RANGES[time_range] if days > 0 - query = query.where("feeds.created_at > NOW() at time zone 'utc' - '#{days} day'::INTERVAL") + query = query.where("feeds.created_at > NOW() - '#{days} day'::INTERVAL") end # handle type filters diff --git a/ruby/lib/jam_ruby/models/friend_request.rb b/ruby/lib/jam_ruby/models/friend_request.rb index b2fe2f5ba..ca5a8c978 100644 --- a/ruby/lib/jam_ruby/models/friend_request.rb +++ b/ruby/lib/jam_ruby/models/friend_request.rb @@ -34,7 +34,7 @@ module JamRuby ActiveRecord::Base.transaction do friend_request = FriendRequest.find(id) friend_request.status = status - friend_request.updated_at = Time.now.getutc + friend_request.updated_at = Time.now friend_request.save # create both records for this friendship diff --git a/ruby/lib/jam_ruby/models/track.rb b/ruby/lib/jam_ruby/models/track.rb index 6ad15159a..e827914f9 100644 --- a/ruby/lib/jam_ruby/models/track.rb +++ b/ruby/lib/jam_ruby/models/track.rb @@ -151,7 +151,7 @@ module JamRuby track.client_track_id = client_track_id end - track.updated_at = Time.now.getutc + track.updated_at = Time.now track.save return track end diff --git a/ruby/lib/jam_ruby/models/user.rb b/ruby/lib/jam_ruby/models/user.rb index 44147da44..09270fa30 100644 --- a/ruby/lib/jam_ruby/models/user.rb +++ b/ruby/lib/jam_ruby/models/user.rb @@ -561,7 +561,7 @@ module JamRuby self.biography = biography end - self.updated_at = Time.now.getutc + self.updated_at = Time.now self.save end diff --git a/ruby/lib/jam_ruby/resque/icecast_config_writer.rb b/ruby/lib/jam_ruby/resque/icecast_config_writer.rb index c40be7d5a..05ce40a26 100644 --- a/ruby/lib/jam_ruby/resque/icecast_config_writer.rb +++ b/ruby/lib/jam_ruby/resque/icecast_config_writer.rb @@ -21,7 +21,7 @@ module JamRuby def self.queue_jobs_needing_retry # if we haven't seen updated_at be tickled in 5 minutes, but config_changed is still set to TRUE, this record has gotten stale - IcecastServer.find_each(:conditions => "config_changed = 1 AND updated_at < (NOW() at time zone 'utc' - interval '#{APP_CONFIG.icecast_max_missing_check} second')", :batch_size => 100) do |server| + IcecastServer.find_each(:conditions => "config_changed = 1 AND updated_at < (NOW() - interval '#{APP_CONFIG.icecast_max_missing_check} second')", :batch_size => 100) do |server| IcecastConfigWriter.enqueue(server.server_id) end end diff --git a/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb b/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb index 14749829f..4927e6568 100644 --- a/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb +++ b/ruby/lib/jam_ruby/resque/scheduled/icecast_source_check.rb @@ -32,7 +32,7 @@ module JamRuby def run # if we haven't seen updated_at be tickled in 5 minutes, but config_changed is still set to TRUE, this record has gotten stale - IcecastMount.find_each(lock: true, :conditions => "sourced_needs_changing_at < (NOW() at time zone 'utc' - interval '#{APP_CONFIG.icecast_max_sourced_changed} second')", :batch_size => 100) do |mount| + IcecastMount.find_each(lock: true, :conditions => "sourced_needs_changing_at < (NOW() - interval '#{APP_CONFIG.icecast_max_sourced_changed} second')", :batch_size => 100) do |mount| if mount.music_session_id mount.with_lock do handle_notifications(mount) diff --git a/ruby/spec/jam_ruby/connection_manager_spec.rb b/ruby/spec/jam_ruby/connection_manager_spec.rb index 6ae0fc029..2580467f1 100644 --- a/ruby/spec/jam_ruby/connection_manager_spec.rb +++ b/ruby/spec/jam_ruby/connection_manager_spec.rb @@ -4,6 +4,10 @@ require 'spec_helper' describe ConnectionManager do TRACKS = [{"instrument_id" => "electric guitar", "sound" => "mono", "client_track_id" => "some_client_track_id"}] + STALE_TIME = 40 + EXPIRE_TIME = 60 + STALE_BUT_NOT_EXPIRED = 50 + DEFINITELY_EXPIRED = 70 before do @conn = PG::Connection.new(:dbname => SpecDb::TEST_DB_NAME, :user => "postgres", :password => "postgres", :host => "localhost") @@ -53,8 +57,8 @@ describe ConnectionManager do user.save! user = nil - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') - expect { @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') }.to raise_error(PG::Error) + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) + expect { @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) }.to raise_error(PG::Error) end it "create connection then delete it" do @@ -63,7 +67,7 @@ describe ConnectionManager do #user_id = create_user("test", "user2", "user2@jamkazam.com") user = FactoryGirl.create(:user) - count = @connman.create_connection(user.id, client_id, "1.1.1.1", 'client') + count = @connman.create_connection(user.id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) count.should == 1 @@ -98,7 +102,7 @@ describe ConnectionManager do #user_id = create_user("test", "user2", "user2@jamkazam.com") user = FactoryGirl.create(:user) - count = @connman.create_connection(user.id, client_id, "1.1.1.1", 'client') + count = @connman.create_connection(user.id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) count.should == 1 @@ -119,7 +123,7 @@ describe ConnectionManager do cc.region.should eql('TX') cc.countrycode.should eql('US') - @connman.reconnect(cc, nil, "33.1.2.3") + @connman.reconnect(cc, nil, "33.1.2.3", STALE_TIME, EXPIRE_TIME) cc = Connection.find_by_client_id!(client_id) cc.connected?.should be_true @@ -237,57 +241,62 @@ describe ConnectionManager do it "flag stale connection" do client_id = "client_id8" user_id = create_user("test", "user8", "user8@jamkazam.com") - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) num = JamRuby::Connection.count(:conditions => ['aasm_state = ?','connected']) num.should == 1 assert_num_connections(client_id, num) - @connman.flag_stale_connections(60) + @connman.flag_stale_connections() assert_num_connections(client_id, num) - sleep(1) + conn = Connection.find_by_client_id(client_id) + set_updated_at(conn, Time.now - STALE_BUT_NOT_EXPIRED) - num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() at time zone 'utc' - interval '#{1} second') AND aasm_state = 'connected'"]) + num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() - interval '#{1} second') AND aasm_state = 'connected'"]) num.should == 1 # this should change the aasm_state to stale - @connman.flag_stale_connections(1) + @connman.flag_stale_connections() - num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() at time zone 'utc' - interval '#{1} second') AND aasm_state = 'connected'"]) + num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() - interval '#{1} second') AND aasm_state = 'connected'"]) num.should == 0 - num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() at time zone 'utc' - interval '#{1} second') AND aasm_state = 'stale'"]) + num = JamRuby::Connection.count(:conditions => ["updated_at < (NOW() - interval '#{1} second') AND aasm_state = 'stale'"]) num.should == 1 assert_num_connections(client_id, 1) - cids = @connman.stale_connection_client_ids(1) + conn = Connection.find_by_client_id(client_id) + set_updated_at(conn, Time.now - DEFINITELY_EXPIRED) + + cids = @connman.stale_connection_client_ids() cids.size.should == 1 cids[0][:client_id].should == client_id - cids[0][:client_type].should == 'native' + cids[0][:client_type].should == Connection::TYPE_CLIENT cids[0][:music_session_id].should be_nil cids[0][:user_id].should == user_id - cids.each { |cid| @connman.delete_connection(cid) } + cids.each { |cid| @connman.delete_connection(cid[:client_id]) } - sleep(1) assert_num_connections(client_id, 0) end it "expires stale connection" do client_id = "client_id8" user_id = create_user("test", "user8", "user8@jamkazam.com") - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) - sleep(1) - @connman.flag_stale_connections(1) + conn = Connection.find_by_client_id(client_id) + set_updated_at(conn, Time.now - STALE_BUT_NOT_EXPIRED) + + @connman.flag_stale_connections assert_num_connections(client_id, 1) # assert_num_connections(client_id, JamRuby::Connection.count(:conditions => ['aasm_state = ?','stale'])) - @connman.expire_stale_connections(60) + @connman.expire_stale_connections assert_num_connections(client_id, 1) - sleep(1) + set_updated_at(conn, Time.now - DEFINITELY_EXPIRED) # this should delete the stale connection - @connman.expire_stale_connections(1) + @connman.expire_stale_connections assert_num_connections(client_id, 0) end @@ -300,7 +309,7 @@ describe ConnectionManager do user = User.find(user_id) music_session = MusicSession.find(music_session_id) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) connection = @connman.join_music_session(user, client_id, music_session, true, TRACKS) connection.errors.any?.should be_false @@ -336,8 +345,8 @@ describe ConnectionManager do client_id2 = "client_id10.12" user_id = create_user("test", "user10.11", "user10.11@jamkazam.com", :musician => true) user_id2 = create_user("test", "user10.12", "user10.12@jamkazam.com", :musician => false) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') - @connman.create_connection(user_id2, client_id2, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) + @connman.create_connection(user_id2, client_id2, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) music_session_id = create_music_session(user_id) @@ -356,7 +365,7 @@ describe ConnectionManager do it "as_musician is coerced to boolean" do client_id = "client_id10.2" user_id = create_user("test", "user10.2", "user10.2@jamkazam.com", :musician => false) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) music_session_id = create_music_session(user_id) @@ -374,8 +383,8 @@ describe ConnectionManager do fan_client_id = "client_id10.4" musician_id = create_user("test", "user10.3", "user10.3@jamkazam.com") fan_id = create_user("test", "user10.4", "user10.4@jamkazam.com", :musician => false) - @connman.create_connection(musician_id, musician_client_id, "1.1.1.1", 'client') - @connman.create_connection(fan_id, fan_client_id, "1.1.1.1", 'client') + @connman.create_connection(musician_id, musician_client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) + @connman.create_connection(fan_id, fan_client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) music_session_id = create_music_session(musician_id, :fan_access => false) @@ -400,7 +409,7 @@ describe ConnectionManager do user = User.find(user_id2) music_session = MusicSession.find(music_session_id) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) # specify real user id, but not associated with this session expect { @connman.join_music_session(user, client_id, music_session, true, TRACKS) } .to raise_error(ActiveRecord::RecordNotFound) end @@ -412,7 +421,7 @@ describe ConnectionManager do user = User.find(user_id) music_session = MusicSession.new - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) connection = @connman.join_music_session(user, client_id, music_session, true, TRACKS) connection.errors.size.should == 1 connection.errors.get(:music_session).should == [ValidationMessages::MUSIC_SESSION_MUST_BE_SPECIFIED] @@ -427,7 +436,7 @@ describe ConnectionManager do user = User.find(user_id2) music_session = MusicSession.find(music_session_id) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) # specify real user id, but not associated with this session expect { @connman.join_music_session(user, client_id, music_session, true, TRACKS) } .to raise_error(ActiveRecord::RecordNotFound) end @@ -441,7 +450,7 @@ describe ConnectionManager do user = User.find(user_id) dummy_music_session = MusicSession.new - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) expect { @connman.leave_music_session(user, Connection.find_by_client_id(client_id), dummy_music_session) }.to raise_error(JamRuby::StateError) end @@ -457,7 +466,7 @@ describe ConnectionManager do dummy_music_session = MusicSession.new - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) @connman.join_music_session(user, client_id, music_session, true, TRACKS) expect { @connman.leave_music_session(user, Connection.find_by_client_id(client_id), dummy_music_session) }.to raise_error(JamRuby::StateError) end @@ -471,7 +480,7 @@ describe ConnectionManager do user = User.find(user_id) music_session = MusicSession.find(music_session_id) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) @connman.join_music_session(user, client_id, music_session, true, TRACKS) assert_session_exists(music_session_id, true) @@ -501,11 +510,11 @@ describe ConnectionManager do music_session = MusicSession.find(create_music_session(user_id)) client_id = Faker::Number.number(20) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) connection = @connman.join_music_session(user, client_id, music_session, true, TRACKS) client_id = Faker::Number.number(20) - @connman.create_connection(user_id, client_id, Faker::Internet.ip_v4_address, 'client') + @connman.create_connection(user_id, client_id, Faker::Internet.ip_v4_address, 'client', STALE_TIME, EXPIRE_TIME) music_session = MusicSession.find(create_music_session(user_id)) connection = @connman.join_music_session(user, client_id, music_session, true, TRACKS) @@ -514,11 +523,11 @@ describe ConnectionManager do user.update_attribute(:admin, true) client_id = Faker::Number.number(20) - @connman.create_connection(user_id, client_id, "1.1.1.1", 'client') + @connman.create_connection(user_id, client_id, "1.1.1.1", 'client', STALE_TIME, EXPIRE_TIME) music_session = MusicSession.find(create_music_session(user_id)) connection = @connman.join_music_session(user, client_id, music_session, true, TRACKS) client_id = Faker::Number.number(20) - @connman.create_connection(user_id, client_id, Faker::Internet.ip_v4_address, 'client') + @connman.create_connection(user_id, client_id, Faker::Internet.ip_v4_address, 'client', STALE_TIME, EXPIRE_TIME) music_session = MusicSession.find(create_music_session(user_id)) connection = @connman.join_music_session(user, client_id, music_session, true, TRACKS) connection.errors.size.should == 0 diff --git a/ruby/spec/jam_ruby/models/track_spec.rb b/ruby/spec/jam_ruby/models/track_spec.rb index be63c781b..b6263ce8d 100644 --- a/ruby/spec/jam_ruby/models/track_spec.rb +++ b/ruby/spec/jam_ruby/models/track_spec.rb @@ -74,14 +74,7 @@ describe Track do it "updates a single track using .id to correlate" do track.id.should_not be_nil connection.tracks.length.should == 1 - begin - ActiveRecord::Base.record_timestamps = false - track.updated_at = 1.days.ago - track.save! - ensure - # very important to turn it back; it'll break all tests otherwise - ActiveRecord::Base.record_timestamps = true - end + set_updated_at(track, 1.days.ago) tracks = Track.sync(connection.client_id, [{:id => track.id, :client_track_id => 'client_guid_new', :sound => 'mono', :instrument_id => 'drums'}]) tracks.length.should == 1 found = tracks[0] @@ -105,14 +98,7 @@ describe Track do it "does not touch updated_at when nothing changes" do track.id.should_not be_nil connection.tracks.length.should == 1 - begin - ActiveRecord::Base.record_timestamps = false - track.updated_at = 1.days.ago - track.save! - ensure - # very important to turn it back; it'll break all tests otherwise - ActiveRecord::Base.record_timestamps = true - end + set_updated_at(track, 1.days.ago) tracks = Track.sync(connection.client_id, [{:id => track.id, :client_track_id => track.client_track_id, :sound => track.sound, :instrument_id => track.instrument_id}]) tracks.length.should == 1 found = tracks[0] diff --git a/ruby/spec/jam_ruby/resque/icecast_config_worker_spec.rb b/ruby/spec/jam_ruby/resque/icecast_config_worker_spec.rb index e10c5f847..c93363a94 100644 --- a/ruby/spec/jam_ruby/resque/icecast_config_worker_spec.rb +++ b/ruby/spec/jam_ruby/resque/icecast_config_worker_spec.rb @@ -136,14 +136,7 @@ describe IcecastConfigWriter do pending "failing on build server" server.touch - begin - ActiveRecord::Base.record_timestamps = false - server.updated_at = Time.now.ago(APP_CONFIG.icecast_max_missing_check + 1) - server.save! - ensure - # very important to turn it back; it'll break all tests otherwise - ActiveRecord::Base.record_timestamps = true - end + set_updated_at(server, Time.now.ago(APP_CONFIG.icecast_max_missing_check + 1)) # should enqueue 1 job IcecastConfigWriter.queue_jobs_needing_retry diff --git a/ruby/spec/support/utilities.rb b/ruby/spec/support/utilities.rb index 0f78e56c4..23b37a3e6 100644 --- a/ruby/spec/support/utilities.rb +++ b/ruby/spec/support/utilities.rb @@ -122,6 +122,18 @@ def run_tests? type ENV["RUN_#{type}_TESTS"] == "1" || ENV[type] == "1" || ENV['ALL_TESTS'] == "1" end +# you have go out of your way to update 'updated_at ' +def set_updated_at(resource, time) + begin + ActiveRecord::Base.record_timestamps = false + resource.updated_at = time + resource.save!(validate: false) + ensure + # very important to turn it back; it'll break all tests otherwise + ActiveRecord::Base.record_timestamps = true + end +end + def wipe_s3_test_bucket # don't bother if the user isn't doing AWS tests if run_tests? :aws diff --git a/web/app/assets/javascripts/JamServer.js b/web/app/assets/javascripts/JamServer.js index 077df1392..8f3a6c2ef 100644 --- a/web/app/assets/javascripts/JamServer.js +++ b/web/app/assets/javascripts/JamServer.js @@ -188,7 +188,7 @@ heartbeatMS = payload.heartbeat_interval * 1000; connection_expire_time = payload.connection_expire_time * 1000; - logger.debug("jamkazam.js.loggedIn(): clientId=" + app.clientId + ", heartbeat=" + heartbeatMS + "ms, expire_time=" + connection_expire_time); + logger.debug("jamkazam.js.loggedIn(): clientId=" + app.clientId + ", heartbeat=" + payload.heartbeat_interval + "s, expire_time=" + payload.connection_expire_time + 's'); heartbeatInterval = context.setInterval(_heartbeat, heartbeatMS); heartbeatAckCheckInterval = context.setInterval(_heartbeatAckCheck, 1000); lastHeartbeatAckTime = new Date(new Date().getTime() + heartbeatMS); // add a little forgiveness to server for initial heartbeat @@ -251,6 +251,15 @@ function performReconnect() { + if(!clientClosedConnection) { + lastDisconnectedReason = 'WEBSOCKET_CLOSED_REMOTELY' + clientClosedConnection = false; + } + else if(!lastDisconnectedReason) { + // let's have at least some sort of type, however generci + lastDisconnectedReason = 'WEBSOCKET_CLOSED_LOCALLY' + } + rest.createDiagnostic({ type: lastDisconnectedReason, data: {logs: logger.logCache, client_type: clientType, client_id: server.clientID, channel_id: channelId} @@ -523,10 +532,6 @@ connectDeferred.reject(); } - if(!clientClosedConnection) { - lastDisconnectedReason = 'WEBSOCKET_CLOSED_REMOTELY' - clientClosedConnection = false; - } closedCleanup(true); }; diff --git a/web/config/application.rb b/web/config/application.rb index aa2078e65..7dff3b4c1 100644 --- a/web/config/application.rb +++ b/web/config/application.rb @@ -104,17 +104,12 @@ if defined?(Bundler) # Websocket-gateway embedded configs config.websocket_gateway_enable = false - if Rails.env=='test' - config.websocket_gateway_connect_time_stale_client = 4 - config.websocket_gateway_connect_time_expire_client = 6 - config.websocket_gateway_connect_time_stale_browser = 4 - config.websocket_gateway_connect_time_expire_browser = 6 - else - config.websocket_gateway_connect_time_stale_client = 40 # 40 matches production - config.websocket_gateway_connect_time_expire_client = 60 # 60 matches production - config.websocket_gateway_connect_time_stale_browser = 40 # 40 matches production - config.websocket_gateway_connect_time_expire_browser = 60 # 60 matches production - end + + config.websocket_gateway_connect_time_stale_client = 40 # 40 matches production + config.websocket_gateway_connect_time_expire_client = 60 # 60 matches production + config.websocket_gateway_connect_time_stale_browser = 40 # 40 matches production + config.websocket_gateway_connect_time_expire_browser = 60 # 60 matches production + config.websocket_gateway_internal_debug = false config.websocket_gateway_port = 6767 + ENV['JAM_INSTANCE'].to_i # Runs the websocket gateway within the web app diff --git a/web/config/environments/test.rb b/web/config/environments/test.rb index b3f417864..81b4884e6 100644 --- a/web/config/environments/test.rb +++ b/web/config/environments/test.rb @@ -47,6 +47,11 @@ SampleApp::Application.configure do config.websocket_gateway_port = 6769 config.websocket_gateway_uri = "ws://localhost:#{config.websocket_gateway_port}/websocket" + config.websocket_gateway_connect_time_stale_client = 4 + config.websocket_gateway_connect_time_expire_client = 6 + config.websocket_gateway_connect_time_stale_browser = 4 + config.websocket_gateway_connect_time_expire_browser = 6 + # this is totally awful and silly; the reason this exists is so that if you upload an artifact # through jam-admin, then jam-web can point users at it. I think 99% of devs won't even see or care about this config, and 0% of users config.jam_admin_root_url = 'http://localhost:3333' diff --git a/web/spec/features/reconnect_spec.rb b/web/spec/features/reconnect_spec.rb index d7d5f11ae..ce822ba51 100644 --- a/web/spec/features/reconnect_spec.rb +++ b/web/spec/features/reconnect_spec.rb @@ -13,6 +13,7 @@ describe "Reconnect", :js => true, :type => :feature, :capybara_feature => true end before(:each) do + Diagnostic.delete_all emulate_client end @@ -62,7 +63,7 @@ describe "Reconnect", :js => true, :type => :feature, :capybara_feature => true sign_in_poltergeist(user1) - 5.times do + 5.times do |i| close_websocket # we should see indication that the websocket is down @@ -70,6 +71,11 @@ describe "Reconnect", :js => true, :type => :feature, :capybara_feature => true # but.. after a few seconds, it should reconnect on it's own page.should_not have_selector('.no-websocket-connection') + + # confirm that a diagnostic was written + Diagnostic.count.should == i + 1 + diagnostic = Diagnostic.first + diagnostic.type.should == Diagnostic::WEBSOCKET_CLOSED_LOCALLY end # then verify we can create a session diff --git a/web/spec/requests/diagnostics_api_spec.rb b/web/spec/requests/diagnostics_api_spec.rb index ca677866d..4788e6996 100644 --- a/web/spec/requests/diagnostics_api_spec.rb +++ b/web/spec/requests/diagnostics_api_spec.rb @@ -28,7 +28,7 @@ describe "Diagnostics", :type => :api do it "can fail" do post "/api/diagnostics.json", {}.to_json, "CONTENT_TYPE" => 'application/json' last_response.status.should eql(422) - JSON.parse(last_response.body).should eql({"errors"=>{"type"=>["is not included in the list"], "creator"=>["is not included in the list"]}}) + JSON.parse(last_response.body).should eql({"errors"=>{"type"=>["is not included in the list"]}}) Diagnostic.count.should == 0 end diff --git a/websocket-gateway/config/application.yml b/websocket-gateway/config/application.yml index 1b5e37f1b..039a2316a 100644 --- a/websocket-gateway/config/application.yml +++ b/websocket-gateway/config/application.yml @@ -1,8 +1,8 @@ Defaults: &defaults connect_time_stale_client: 40 - connect_time_expire_client: 62 + connect_time_expire_client: 60 connect_time_stale_browser: 40 - connect_time_expire_browser: 62 + connect_time_expire_browser: 60 development: port: 6767 diff --git a/websocket-gateway/lib/jam_websockets/client_context.rb b/websocket-gateway/lib/jam_websockets/client_context.rb index 2c8c027cb..73506a369 100644 --- a/websocket-gateway/lib/jam_websockets/client_context.rb +++ b/websocket-gateway/lib/jam_websockets/client_context.rb @@ -15,11 +15,11 @@ end def to_s - return "Client[user:#{@user} client:#{@client} msgs:#{@msg_count} session:#{@session}]" + return "Client[user:#{@user} client:#{@client.client_id} msgs:#{@msg_count} session:#{@session} client_type:#{@client_type} channel_id: #{@client.channel_id}]" end def to_json - {user_id: @user.id, client_id: @client.client_id, msg_count: @msg_count, client_type: @client_type, socket_id: @client.socket_id}.to_json + {user_id: @user.id, client_id: @client.client_id, msg_count: @msg_count, client_type: @client_type, socket_id: @client.channel_id}.to_json end def hash diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index 8ddb254c4..526d413fb 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -35,7 +35,8 @@ module JamWebsockets class Router - attr_accessor :user_context_lookup + attr_accessor :user_context_lookup, :heartbeat_interval_client, :connect_time_expire_client, :connect_time_stale_client, + :heartbeat_interval_browser, :connect_time_expire_browser, :connect_time_stale_browser def initialize() @log = Logging.logger[self] @@ -243,8 +244,7 @@ module JamWebsockets } client.onclose { - @log.debug "Connection closed" - stale_client(client) + @log.debug "connection closed. marking stale: #{client.context}" cleanup_client(client) } @@ -336,6 +336,7 @@ module JamWebsockets # caused a client connection to be marked stale def stale_client(client) if client.client_id + @log.info "marking client stale: #{client.context}" ConnectionManager.active_record_transaction do |connection_manager| music_session_id = connection_manager.flag_connection_stale_with_client_id(client.client_id) # update the session members, letting them know this client went stale @@ -353,29 +354,35 @@ module JamWebsockets client_context = @client_lookup[cid] - diagnostic_data = client_context.to_json unless client_context.nil? - cleanup_client(client_context.client) unless client_context.nil? + if client_context + diagnostic_data = client_context.to_json + cleanup_client(client_context.client) + Diagnostic.expired_stale_connection(user_id, diagnostic_data) + end + music_session = nil - recordingId = nil + recording_id = nil user = nil # remove this connection from the database ConnectionManager.active_record_transaction do |mgr| mgr.delete_connection(cid) { |conn, count, music_session_id, user_id| - Diagnostic.expired_stale_connection(user_id, diagnostic_data) + @log.info "expiring stale connection client_id:#{cid}, user_id:#{user_id}" Notification.send_friend_update(user_id, false, conn) if count == 0 music_session = MusicSession.find_by_id(music_session_id) unless music_session_id.nil? user = User.find_by_id(user_id) unless user_id.nil? recording = music_session.stop_recording unless music_session.nil? # stop any ongoing recording, if there is one - recordingId = recording.id unless recording.nil? + recording_id = recording.id unless recording.nil? music_session.with_lock do # VRFS-1297 music_session.tick_track_changes end if music_session } end - Notification.send_session_depart(music_session, cid, user, recordingId) unless music_session.nil? || user.nil? + if user && music_session + Notification.send_session_depart(music_session, cid, user, recording_id) + end end end @@ -511,6 +518,7 @@ module JamWebsockets existing_context = @client_lookup[client_id] if existing_context # in some reconnect scenarios, we may have in memory a websocket client still. + @log.info "duplicate client: #{existing_context}" Diagnostic.duplicate_client(existing_context.user, existing_context) if existing_context.client.connected cleanup_client(existing_context.client) end @@ -532,7 +540,7 @@ module JamWebsockets heartbeat_interval, connection_stale_time, connection_expire_time = determine_connection_times(user, client_type) - @log.debug "user #{user} logged in with client_id #{client_id}" + @log.debug "logged in #{user} with client_id: #{client_id}" # check if there's a connection for the client... if it's stale, reconnect it unless connection.nil? @@ -579,6 +587,8 @@ module JamWebsockets add_user(context) add_client(client_id, context) + @log.debug "logged in context created: #{context}" + unless connection # log this connection in the database ConnectionManager.active_record_transaction do |connection_manager| @@ -828,11 +838,15 @@ module JamWebsockets def sane_logging(&blk) # used around repeated transactions that cause too much ActiveRecord::Base logging begin - original_level = @ar_base_logger.level - @ar_base_logger.level = :info if @ar_base_logger + if @ar_base_logger + original_level = @ar_base_logger.level + @ar_base_logger.level = :info + end blk.call ensure - @ar_base_logger.level = original_level if @ar_base_logger + if @ar_base_logger + @ar_base_logger.level = original_level + end end end end diff --git a/websocket-gateway/lib/jam_websockets/server.rb b/websocket-gateway/lib/jam_websockets/server.rb index 4aafa62c2..200bcfe8b 100644 --- a/websocket-gateway/lib/jam_websockets/server.rb +++ b/websocket-gateway/lib/jam_websockets/server.rb @@ -33,8 +33,8 @@ module JamWebsockets EventMachine.run do @router.start(connect_time_stale_client, connect_time_expire_client, connect_time_stale_browser, connect_time_expire_browser, host: rabbitmq_host, port: rabbitmq_port) do - start_connection_expiration() - start_connection_flagger() + start_connection_expiration + start_connection_flagger start_websocket_listener(host, port, options[:emwebsocket_debug]) calling_thread.wakeup if calling_thread end @@ -59,44 +59,45 @@ module JamWebsockets @log.debug("started websocket") end - def start_connection_expiration() + def start_connection_expiration # one cleanup on startup - expire_stale_connections() + expire_stale_connections - EventMachine::PeriodicTimer.new(5) do - sane_logging { expire_stale_connections() } + EventMachine::PeriodicTimer.new(2) do + sane_logging { expire_stale_connections } end end - def expire_stale_connections() + def expire_stale_connections clients = [] ConnectionManager.active_record_transaction do |connection_manager| - clients = connection_manager.stale_connection_client_ids() + clients = connection_manager.stale_connection_client_ids end @router.cleanup_clients_with_ids(clients) end - def start_connection_flagger() + def start_connection_flagger # one cleanup on startup - flag_stale_connections() + flag_stale_connections - EventMachine::PeriodicTimer.new(5) do - sane_logging { flag_stale_connections() } + EventMachine::PeriodicTimer.new(2) do + sane_logging { flag_stale_connections } end end def flag_stale_connections() # @log.debug("*** flag_stale_connections: fires each #{flag_max_time} seconds") ConnectionManager.active_record_transaction do |connection_manager| - connection_manager.flag_stale_connections() + connection_manager.flag_stale_connections end end def sane_logging(&blk) # used around repeated transactions that cause too much ActiveRecord::Base logging + # example is handling heartbeats begin - original_level = @ar_base_logger.level + original_level = @ar_base_logger.level if @ar_base_logger @ar_base_logger.level = :info if @ar_base_logger blk.call ensure diff --git a/websocket-gateway/spec/jam_websockets/client_context_spec.rb b/websocket-gateway/spec/jam_websockets/client_context_spec.rb index 47f422ff4..b5f17e22b 100644 --- a/websocket-gateway/spec/jam_websockets/client_context_spec.rb +++ b/websocket-gateway/spec/jam_websockets/client_context_spec.rb @@ -2,7 +2,13 @@ require 'spec_helper' describe ClientContext do - let(:context) {ClientContext.new({}, "client1", "client")} + let(:client) { + fake_client = double(Object) + fake_client.should_receive(:context=).any_number_of_times + fake_client.should_receive(:context).any_number_of_times + fake_client + } + let(:context) {ClientContext.new({}, client, "client")} describe 'hashing' do it "hash correctly" do diff --git a/websocket-gateway/spec/jam_websockets/router_spec.rb b/websocket-gateway/spec/jam_websockets/router_spec.rb index e1df86433..094702158 100644 --- a/websocket-gateway/spec/jam_websockets/router_spec.rb +++ b/websocket-gateway/spec/jam_websockets/router_spec.rb @@ -2,13 +2,17 @@ require 'spec_helper' require 'thread' LoginClient = Class.new do - attr_accessor :onmsgblock, :onopenblock, :encode_json, :client_id + attr_accessor :onmsgblock, :onopenblock, :encode_json, :channel_id, :client_id, :user_id, :context def initialize() end + def connected? + true + end + def onopen(&block) @onopenblock = block end @@ -57,7 +61,7 @@ def login(router, user, password, client_id) @router.new_client(client) handshake = double("handshake") - handshake.should_receive(:query).and_return({ "pb" => "true" }) + handshake.should_receive(:query).twice.and_return({ "pb" => "true", "channel_id" => SecureRandom.uuid }) client.onopenblock.call handshake # create a login message, and pass it into the router via onmsgblock.call @@ -89,6 +93,12 @@ describe Router do em_before do @router = Router.new() + @router.connect_time_expire_client = 60 + @router.connect_time_stale_client = 40 + @router.heartbeat_interval_client = @router.connect_time_stale_client / 2 + @router.connect_time_expire_browser = 60 + @router.connect_time_stale_browser = 40 + @router.heartbeat_interval_browser = @router.connect_time_stale_browser / 2 end subject { @router } @@ -126,6 +136,7 @@ describe Router do user = double(User) user.should_receive(:id).any_number_of_times.and_return("1") client = double("client") + client.should_receive(:context=).any_number_of_times context = ClientContext.new(user, client, "client") @router.user_context_lookup.length.should == 0 From 53850a716b430fa846f547d7735564d4b41c9f97 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Fri, 2 May 2014 07:33:41 -0500 Subject: [PATCH 17/20] * fixing bad ref to user_id in client cleanup code --- ruby/lib/jam_ruby/models/diagnostic.rb | 4 ++-- websocket-gateway/lib/jam_websockets/router.rb | 4 +--- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/ruby/lib/jam_ruby/models/diagnostic.rb b/ruby/lib/jam_ruby/models/diagnostic.rb index fa6c31867..a4be2dab5 100644 --- a/ruby/lib/jam_ruby/models/diagnostic.rb +++ b/ruby/lib/jam_ruby/models/diagnostic.rb @@ -46,8 +46,8 @@ module JamRuby validates :data, length: {maximum: 100000} - def self.expired_stale_connection(user, context_as_json) - Diagnostic.save(EXPIRED_STALE_CONNECTION, user, WEBSOCKET_GATEWAY, context_as_json) if user + def self.expired_stale_connection(user, context) + Diagnostic.save(EXPIRED_STALE_CONNECTION, user, WEBSOCKET_GATEWAY, context.to_json) if user end def self.missing_client_state(user, context) diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index 526d413fb..c11bc9c89 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -355,12 +355,10 @@ module JamWebsockets client_context = @client_lookup[cid] if client_context - diagnostic_data = client_context.to_json + Diagnostic.expired_stale_connection(client_context.user.id, client_context) cleanup_client(client_context.client) - Diagnostic.expired_stale_connection(user_id, diagnostic_data) end - music_session = nil recording_id = nil user = nil From 14d6904162641af0892565ce4a81fe84d6c6bad7 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Fri, 2 May 2014 07:39:09 -0500 Subject: [PATCH 18/20] * fixing another bug where .reconnect is not supplied the correct arguments --- websocket-gateway/lib/jam_websockets/router.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index c11bc9c89..dceb79288 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -642,8 +642,10 @@ module JamWebsockets update_notification_seen_at(connection, context, heartbeat) end + ConnectionManager.active_record_transaction do |connection_manager| - connection_manager.reconnect(connection, connection.music_session_id, nil) + heartbeat_interval, connection_stale_time, connection_expire_time = determine_connection_times(context.user, client_type) + connection_manager.reconnect(connection, connection.music_session_id, nil, connection_stale_time, connection_expire_time) end if connection.stale? end From 30b3098244b9f2fb3dfdb4325d92dcb6749a64fa Mon Sep 17 00:00:00 2001 From: Seth Call Date: Fri, 2 May 2014 19:07:10 -0500 Subject: [PATCH 19/20] * don't null user on context; it should always be non-null --- websocket-gateway/lib/jam_websockets/router.rb | 2 -- 1 file changed, 2 deletions(-) diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index dceb79288..3b424517c 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -125,8 +125,6 @@ module JamWebsockets if user_contexts.length == 0 @user_context_lookup.delete(client_context.user.id) end - - client_context.user = nil end end From 15936855e965375ba2a2e5e28d33e799bbb471c3 Mon Sep 17 00:00:00 2001 From: Seth Call Date: Sat, 3 May 2014 08:50:06 -0500 Subject: [PATCH 20/20] * fixing yet another bug associated with websockets and reconnect --- websocket-gateway/lib/jam_websockets/router.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/websocket-gateway/lib/jam_websockets/router.rb b/websocket-gateway/lib/jam_websockets/router.rb index 3b424517c..33f719fcf 100644 --- a/websocket-gateway/lib/jam_websockets/router.rb +++ b/websocket-gateway/lib/jam_websockets/router.rb @@ -642,7 +642,7 @@ module JamWebsockets ConnectionManager.active_record_transaction do |connection_manager| - heartbeat_interval, connection_stale_time, connection_expire_time = determine_connection_times(context.user, client_type) + heartbeat_interval, connection_stale_time, connection_expire_time = determine_connection_times(context.user, context.client_type) connection_manager.reconnect(connection, connection.music_session_id, nil, connection_stale_time, connection_expire_time) end if connection.stale? end