5

UPDATE: I think I was over-complicating my problem and am struggling through a new approach as described here: paired t-test as a simple latent change score model

I am still accepting the answer below as it does appear the model is unidentified.

If folks think this is better placed on Stack Overflow, please advise. lavaan questions don't seem to get picked up too frequently there, so I'm starting here.

I am trying to estimate a variation of a change score model as described in the following powerpoint: http://davidakenny.net/webinars/powerpoints/SEM/Longitudinal/Change/CSA.ppt

I don't know the author and am a bit of a novice at latent variable modeling, so I am happy to receive advice on this approach in general...

The path diagram I think I am estimating is as follows:

enter image description here

NOTE 1: I understand that a two-wave follow-up is not preferred for growth-modeling, but this is a limitation of my data.

NOTE 2: I only have two indicator measures instead of three as shown in the presentation. I am thus fixing all of the loadings to 1 in order to accommodate this and ensure the model is identified.

In laavan, I am specifying the model as follows:

LCSM_Raykov <- '
B=~ Y_11 + 1*Y_12 + 1*Y_21 + 1*Y_22
C=~ Y_21 + 1*Y_22
# regressions
C ~ X
# residual correlations
B ~~ X
B ~~ C
Y_11 ~~ Y_12
Y_21 ~~ Y_22
'

LCSM_Raykov_fit <- lavaan(LCSM_Raykov, data=dat)

However, I am receiving the following error message.

#Error in lav_model_estimate(lavmodel = lavmodel, lavsamplestats = #lavsamplestats,  : 
#  lavaan ERROR: initial model-implied matrix (Sigma) is not positive #definite;
#  check your model and/or starting parameters.
#In addition: Warning message:
#In vnames(FLAT, "ov.x", warn = TRUE) :
#  lavaan WARNING: model syntax contains variance/covariance/intercept #formulas
#  involving (an) exogenous variable(s): [X];
#  Please use fixed.x=FALSE or leave them alone

I am curious as to whether or not this is a problem with my data (which I cannot share) or if it is a problem with my specification.

A simulated set of data which reproduces the error is provided below:

dat <- structure(list(Y_11 = c(420, 354, 308, 415, 373, 354, 342, 434, 
327, 395, 315, 342, 367, 330, 436, 318, 460, 424, 384, 363, 384, 
367, 400, 333, 401, 354, 364, 369, 358, 373, 361, 384, 401, 435, 
368, 433, 360, 388, 373, 383, 349, 371, 400, 315, 342, 411, 360, 
375, 424, 326, 383, 380, 320, 424, 387, 393, 401, 366, 378, 448, 
370, 392, 414, 363, 342, 358, 411, 358, 378, 358, 388, 350, 447, 
371, 385, 361, 308, 357, 367, 371, 389, 397, 415, 370, 393, 327, 
414, 352, 417, 376, 488, 361, 384, 435, 454, 309, 407, 387, 434, 
313, 476, 409, 349, 366, 308, 333, 373, 395, 407, 404, 407, 358, 
363, 337, 354, 408, 337, 460, 378, 378, 394, 354, 352, 376, 407, 
364, 381, 443, 414, 346, 401, 373, 408, 342, 352, 400, 361, 417, 
384, 426, 364, 492, 376, 404, 333, 396, 381, 302, 433, 341, 303, 
358, 404, 337, 368, 433, 338, 407, 366, 363, 377, 338, 344, 442, 
352, 373, 380, 346, 388, 357, 397, 428, 329, 358, 354, 417, 346, 
388, 381, 378, 383, 344, 424, 373, 387, 318, 428, 435, 414, 396, 
396, 371, 348, 345, 344, 414, 378, 392, 378, 366, 460, 320, 342, 
407, 337, 326, 313, 345, 396, 351, 354, 417, 431, 467, 393, 414, 
391, 369, 303, 338, 417, 414, 358, 432, 400, 456, 411, 400, 376, 
296, 360, 327, 417, 338, 321, 388, 420, 409, 396, 400, 407, 338, 
411, 397, 394, 381, 397, 357, 381, 341, 407, 315, 421, 404, 388, 
378, 435, 467, 303, 337, 360, 261, 396, 361, 363, 430, 382, 411, 
370, 357, 404, 375, 400, 400, 381, 313, 421, 412, 341, 394, 385, 
337, 354, 341, 366, 338, 406, 378, 348, 368, 337, 483, 329, 364, 
308, 369, 437, 388, 315, 460, 382, 361, 414, 420, 355, 355, 330, 
395, 373, 321, 396, 373, 395, 373, 357, 421, 370, 379, 394, 375, 
344, 358, 407, 488, 376, 366, 412, 321, 400, 443, 391, 391, 420, 
388, 387, 330, 334, 338, 387, 361, 433, 394, 414, 373, 363, 384, 
346, 326, 417, 327, 373, 403, 375, 431, 303, 371, 381, 402, 378, 
401, 477, 384, 389, 370, 423, 383, 352, 338, 381, 349, 303, 329, 
379, 417, 400, 355, 391, 408, 420), Y_12 = c(459, 364, 358, 409, 
412, 381, 375, 412, 383, 445, 386, 397, 404, 363, 436, 343, 467, 
409, 401, 395, 412, 402, 431, 376, 394, 406, 415, 406, 370, 404, 
404, 415, 431, 422, 397, 428, 375, 406, 361, 412, 404, 372, 422, 
324, 364, 430, 378, 388, 412, 392, 404, 386, 375, 409, 401, 412, 
432, 384, 426, 412, 373, 401, 404, 384, 368, 392, 432, 388, 393, 
404, 386, 364, 452, 366, 412, 404, 346, 401, 412, 340, 404, 409, 
415, 402, 397, 364, 426, 366, 437, 414, 467, 375, 395, 430, 436, 
394, 397, 436, 427, 349, 436, 412, 349, 376, 358, 367, 402, 407, 
397, 412, 400, 363, 376, 376, 412, 445, 392, 420, 388, 400, 470, 
375, 390, 404, 406, 395, 400, 415, 412, 386, 419, 394, 402, 397, 
366, 402, 400, 432, 412, 422, 409, 437, 383, 428, 370, 417, 406, 
346, 412, 378, 375, 385, 412, 375, 404, 432, 349, 420, 393, 409, 
409, 382, 399, 437, 382, 420, 393, 385, 397, 412, 392, 444, 386, 
358, 390, 436, 381, 404, 402, 331, 419, 399, 427, 414, 402, 358, 
409, 430, 392, 428, 384, 404, 401, 409, 364, 404, 395, 404, 397, 
388, 459, 372, 406, 412, 376, 366, 377, 384, 390, 380, 409, 390, 
430, 428, 417, 392, 392, 404, 345, 392, 426, 414, 390, 422, 412, 
404, 420, 400, 422, 352, 394, 350, 424, 370, 383, 400, 412, 426, 
409, 415, 414, 370, 432, 422, 406, 409, 418, 378, 404, 380, 409, 
357, 420, 415, 406, 407, 436, 483, 359, 392, 412, 323, 414, 397, 
370, 431, 401, 445, 409, 380, 402, 399, 420, 409, 401, 358, 417, 
415, 378, 422, 414, 409, 394, 388, 390, 345, 404, 418, 388, 406, 
404, 444, 386, 412, 355, 409, 397, 397, 350, 424, 401, 400, 418, 
422, 406, 406, 360, 412, 417, 404, 397, 400, 426, 407, 390, 402, 
366, 399, 409, 422, 431, 373, 412, 432, 414, 381, 419, 373, 419, 
483, 409, 409, 409, 397, 397, 401, 368, 353, 381, 388, 404, 406, 
426, 376, 372, 404, 381, 366, 426, 358, 388, 404, 401, 436, 316, 
406, 409, 384, 395, 393, 437, 422, 395, 390, 436, 400, 390, 400, 
404, 394, 375, 390, 401, 412, 424, 397, 404, 412, 395), Y_21 = c(412, 
356, 307, 379, 337, 383, 326, 396, 304, 365, 358, 321, 363, 337, 
464, 314, 375, 383, 366, 356, 387, 330, 386, 310, 386, 368, 342, 
330, 316, 400, 347, 397, 396, 394, 316, 389, 368, 360, 297, 351, 
326, 314, 310, 310, 360, 412, 347, 375, 415, 348, 319, 351, 344, 
415, 355, 415, 396, 312, 380, 417, 321, 365, 385, 377, 358, 357, 
432, 417, 375, 330, 368, 330, 433, 325, 379, 390, 298, 344, 338, 
331, 378, 371, 408, 355, 386, 310, 400, 361, 375, 379, 471, 357, 
363, 380, 433, 353, 396, 394, 442, 325, 417, 382, 312, 307, 344, 
373, 325, 380, 386, 410, 382, 312, 347, 321, 368, 387, 355, 426, 
394, 412, 384, 321, 312, 379, 402, 352, 375, 409, 409, 342, 408, 
348, 394, 307, 344, 386, 334, 422, 378, 445, 365, 478, 330, 489, 
338, 392, 286, 380, 421, 331, 380, 360, 433, 316, 316, 405, 312, 
386, 274, 364, 366, 344, 347, 471, 375, 351, 415, 326, 351, 347, 
383, 433, 365, 321, 378, 422, 330, 375, 352, 310, 384, 325, 485, 
368, 387, 336, 446, 392, 459, 402, 389, 384, 368, 340, 330, 417, 
382, 373, 382, 389, 445, 321, 381, 348, 375, 295, 284, 307, 375, 
350, 373, 403, 449, 442, 364, 400, 379, 350, 344, 325, 407, 382, 
304, 404, 337, 443, 415, 375, 387, 307, 358, 316, 437, 316, 297, 
319, 401, 408, 368, 380, 382, 316, 437, 415, 371, 385, 397, 368, 
357, 377, 427, 422, 419, 411, 368, 365, 415, 433, 320, 358, 334, 
314, 415, 338, 361, 377, 387, 412, 314, 361, 452, 341, 400, 362, 
368, 350, 489, 421, 330, 401, 392, 375, 326, 391, 325, 356, 397, 
385, 330, 331, 321, 403, 375, 366, 326, 380, 432, 350, 330, 360, 
379, 347, 380, 412, 336, 336, 347, 440, 360, 338, 364, 360, 371, 
360, 375, 422, 340, 347, 392, 384, 300, 310, 396, 489, 312, 312, 
418, 338, 348, 437, 371, 377, 438, 336, 400, 298, 286, 284, 321, 
336, 432, 413, 392, 381, 370, 384, 386, 316, 407, 331, 352, 380, 
361, 371, 331, 356, 375, 348, 368, 389, 463, 387, 390, 368, 389, 
379, 221, 342, 387, 298, 380, 337, 366, 380, 405, 344, 379, 383, 
412), Y_22 = c(431, 403, 348, 404, 393, 397, 383, 409, 342, 405, 
412, 406, 388, 356, 447, 370, 381, 390, 350, 375, 390, 359, 409, 
354, 401, 406, 378, 390, 371, 406, 369, 408, 427, 397, 355, 406, 
391, 396, 339, 365, 400, 345, 384, 369, 377, 418, 396, 394, 402, 
428, 406, 371, 415, 411, 407, 408, 422, 348, 394, 431, 359, 412, 
402, 437, 400, 386, 427, 405, 398, 357, 432, 363, 422, 353, 400, 
397, 329, 387, 375, 351, 388, 432, 436, 363, 379, 359, 403, 402, 
411, 396, 427, 361, 384, 419, 418, 384, 433, 405, 427, 365, 441, 
422, 339, 372, 379, 382, 398, 377, 409, 418, 393, 345, 370, 404, 
400, 411, 393, 400, 369, 398, 404, 388, 385, 381, 403, 386, 417, 
419, 422, 367, 442, 404, 418, 372, 396, 411, 377, 403, 408, 462, 
405, 432, 353, 466, 380, 400, 418, 415, 421, 383, 417, 394, 398, 
366, 365, 398, 342, 383, 365, 391, 428, 396, 402, 450, 384, 403, 
391, 390, 373, 417, 418, 406, 396, 343, 390, 433, 388, 400, 371, 
345, 448, 404, 432, 393, 384, 380, 398, 402, 402, 415, 391, 402, 
414, 442, 385, 402, 381, 422, 387, 391, 397, 375, 421, 393, 400, 
368, 348, 378, 385, 394, 411, 396, 456, 422, 406, 382, 421, 422, 
360, 412, 419, 391, 353, 442, 385, 464, 408, 389, 394, 324, 395, 
345, 422, 365, 367, 358, 427, 418, 385, 402, 383, 355, 415, 402, 
428, 397, 394, 412, 373, 442, 418, 429, 411, 417, 389, 405, 402, 
405, 324, 400, 422, 352, 415, 378, 407, 448, 400, 405, 497, 422, 
406, 414, 408, 412, 406, 380, 427, 455, 403, 447, 385, 403, 393, 
406, 387, 345, 390, 418, 378, 383, 407, 448, 400, 412, 314, 415, 
408, 412, 336, 379, 428, 377, 418, 394, 415, 415, 356, 419, 385, 
357, 391, 392, 394, 400, 427, 406, 383, 420, 396, 448, 401, 377, 
389, 466, 373, 389, 388, 392, 415, 441, 391, 412, 418, 393, 437, 
329, 340, 345, 339, 378, 415, 432, 425, 406, 379, 418, 391, 386, 
425, 369, 367, 411, 411, 392, 324, 360, 397, 400, 408, 440, 433, 
397, 425, 392, 437, 393, 391, 379, 402, 448, 417, 412, 422, 388, 
406, 412, 396, 402, 402), X = c(4L, 1L, 1L, 16L, 9L, 4L, 1L, 
16L, 16L, 9L, 1L, 25L, 4L, 1L, 25L, 36L, 4L, 1L, 1L, 1L, 9L, 
1L, 4L, 9L, 4L, 25L, 9L, 9L, 4L, 9L, 9L, 36L, 4L, 4L, 25L, 1L, 
4L, 1L, 9L, 16L, 16L, 16L, 1L, 9L, 9L, 4L, 1L, 1L, 4L, 16L, 4L, 
4L, 1L, 9L, 9L, 1L, 9L, 36L, 4L, 4L, 4L, 9L, 9L, 1L, 36L, 4L, 
9L, 4L, 1L, 4L, 9L, 1L, 1L, 1L, 144L, 25L, 9L, 16L, 1L, 4L, 1L, 
4L, 16L, 16L, 9L, 9L, 1L, 16L, 1L, 4L, 25L, 4L, 4L, 1L, 16L, 
4L, 1L, 25L, 36L, 100L, 4L, 16L, 9L, 49L, 1L, 16L, 25L, 9L, 4L, 
1L, 36L, 16L, 1L, 4L, 1L, 4L, 16L, 16L, 9L, 4L, 81L, 9L, 36L, 
1L, 4L, 4L, 4L, 1L, 9L, 81L, 9L, 36L, 1L, 25L, 16L, 36L, 1L, 
4L, 36L, 4L, 1L, 9L, 9L, 4L, 4L, 4L, 100L, 9L, 4L, 16L, 150L, 
9L, 9L, 16L, 4L, 9L, 9L, 9L, 64L, 36L, 49L, 16L, 25L, 1L, 196L, 
4L, 4L, 16L, 9L, 16L, 9L, 9L, 4L, 25L, 16L, 16L, 16L, 1L, 9L, 
25L, 49L, 4L, 16L, 9L, 16L, 64L, 1L, 144L, 9L, 9L, 4L, 9L, 1L, 
9L, 81L, 9L, 4L, 81L, 9L, 9L, 1L, 1L, 1L, 324L, 1L, 49L, 4L, 
25L, 25L, 4L, 16L, 9L, 4L, 9L, 9L, 4L, 1L, 64L, 64L, 4L, 1L, 
9L, 49L, 1L, 4L, 9L, 9L, 4L, 4L, 64L, 9L, 9L, 16L, 4L, 25L, 4L, 
4L, 9L, 81L, 9L, 9L, 25L, 49L, 36L, 9L, 4L, 4L, 100L, 25L, 121L, 
9L, 25L, 1L, 4L, 16L, 4L, 9L, 36L, 144L, 16L, 16L, 16L, 9L, 100L, 
1L, 4L, 1L, 4L, 81L, 1L, 9L, 64L, 4L, 1L, 9L, 144L, 4L, 16L, 
16L, 4L, 1L, 16L, 225L, 1L, 9L, 9L, 9L, 16L, 121L, 4L, 9L, 4L, 
4L, 16L, 9L, 9L, 9L, 144L, 16L, 9L, 4L, 4L, 25L, 4L, 95L, 266L, 
4L, 4L, 4L, 1L, 25L, 9L, 1L, 1L, 4L, 1L, 144L, 4L, 1L, 9L, 1L, 
49L, 1L, 4L, 25L, 49L, 4L, 25L, 4L, 36L, 1L, 1L, 49L, 144L, 16L, 
9L, 100L, 4L, 36L, 100L, 4L, 4L, 9L, 4L, 9L, 9L, 1L, 49L, 9L, 
4L, 4L, 25L, 4L, 9L, 9L, 1L, 49L, 1L, 196L, 16L, 9L, 16L, 1L, 
4L, 1L, 9L, 4L, 1L, 1L, 4L, 75L, 121L, 1L, 9L, 1L, 4L, 49L, 36L, 
4L)), .Names = c("Y_11", "Y_12", "Y_21", "Y_22", "X"), row.names = c(1L, 
3L, 4L, 5L, 6L, 7L, 8L, 10L, 11L, 12L, 15L, 16L, 18L, 20L, 22L, 
23L, 24L, 25L, 26L, 27L, 28L, 31L, 32L, 33L, 34L, 36L, 38L, 39L, 
40L, 41L, 42L, 43L, 45L, 46L, 47L, 48L, 51L, 52L, 53L, 54L, 55L, 
56L, 57L, 59L, 60L, 61L, 62L, 63L, 65L, 67L, 68L, 71L, 72L, 73L, 
74L, 75L, 76L, 77L, 80L, 81L, 82L, 83L, 84L, 85L, 86L, 87L, 88L, 
89L, 90L, 91L, 92L, 93L, 94L, 95L, 96L, 97L, 98L, 99L, 100L, 
101L, 102L, 104L, 105L, 107L, 108L, 109L, 110L, 111L, 112L, 114L, 
116L, 117L, 119L, 120L, 121L, 122L, 123L, 124L, 125L, 126L, 127L, 
128L, 129L, 130L, 131L, 132L, 134L, 135L, 136L, 137L, 138L, 139L, 
140L, 141L, 142L, 143L, 144L, 145L, 147L, 148L, 149L, 150L, 151L, 
153L, 155L, 156L, 157L, 158L, 159L, 161L, 163L, 164L, 165L, 166L, 
167L, 168L, 169L, 172L, 176L, 177L, 178L, 179L, 180L, 181L, 182L, 
183L, 184L, 185L, 187L, 188L, 189L, 190L, 191L, 193L, 194L, 195L, 
196L, 197L, 198L, 199L, 200L, 201L, 203L, 204L, 205L, 206L, 207L, 
208L, 209L, 210L, 212L, 214L, 215L, 216L, 217L, 218L, 219L, 220L, 
221L, 222L, 223L, 224L, 225L, 226L, 227L, 228L, 229L, 230L, 231L, 
232L, 236L, 237L, 238L, 239L, 240L, 241L, 242L, 244L, 245L, 247L, 
248L, 249L, 250L, 251L, 252L, 253L, 254L, 255L, 256L, 257L, 258L, 
260L, 261L, 262L, 264L, 265L, 266L, 268L, 269L, 270L, 272L, 273L, 
274L, 276L, 279L, 280L, 281L, 282L, 283L, 284L, 285L, 286L, 288L, 
289L, 290L, 291L, 292L, 293L, 294L, 296L, 297L, 298L, 300L, 301L, 
303L, 304L, 305L, 306L, 307L, 308L, 309L, 310L, 311L, 312L, 313L, 
314L, 315L, 316L, 317L, 318L, 319L, 321L, 322L, 323L, 325L, 327L, 
328L, 329L, 330L, 332L, 333L, 334L, 335L, 336L, 337L, 338L, 341L, 
342L, 343L, 345L, 346L, 347L, 348L, 349L, 350L, 351L, 353L, 355L, 
356L, 358L, 359L, 361L, 362L, 363L, 364L, 365L, 366L, 369L, 370L, 
372L, 373L, 374L, 375L, 376L, 378L, 379L, 382L, 383L, 384L, 385L, 
387L, 388L, 389L, 390L, 391L, 393L, 394L, 395L, 396L, 398L, 400L, 
401L, 402L, 405L, 406L, 407L, 408L, 409L, 410L, 411L, 412L, 413L, 
414L, 415L, 416L, 417L, 418L, 419L, 420L, 421L, 422L, 423L, 424L, 
426L, 427L, 429L, 430L, 431L, 432L, 434L, 435L, 436L, 439L, 440L, 
442L, 443L, 445L, 446L, 447L, 448L, 449L, 451L, 452L, 453L, 454L, 
455L, 457L, 458L, 459L, 460L, 461L, 462L, 463L, 464L, 466L, 467L, 
468L, 469L, 470L), class = "data.frame", na.action = structure(c(2L, 
9L, 13L, 14L, 17L, 19L, 21L, 29L, 30L, 35L, 37L, 44L, 49L, 50L, 
58L, 64L, 66L, 69L, 70L, 78L, 79L, 103L, 106L, 113L, 115L, 118L, 
133L, 146L, 152L, 154L, 160L, 162L, 170L, 171L, 173L, 174L, 175L, 
186L, 192L, 202L, 211L, 213L, 233L, 234L, 235L, 243L, 246L, 259L, 
263L, 267L, 271L, 275L, 277L, 278L, 287L, 295L, 299L, 302L, 320L, 
324L, 326L, 331L, 339L, 340L, 344L, 352L, 354L, 357L, 360L, 367L, 
368L, 371L, 377L, 380L, 381L, 386L, 392L, 397L, 399L, 403L, 404L, 
425L, 428L, 433L, 437L, 438L, 441L, 444L, 450L, 456L, 465L), .Names = c("2", 
"9", "13", "14", "17", "19", "21", "29", "30", "35", "37", "44", 
"49", "50", "58", "64", "66", "69", "70", "78", "79", "103", 
"106", "113", "115", "118", "133", "146", "152", "154", "160", 
"162", "170", "171", "173", "174", "175", "186", "192", "202", 
"211", "213", "233", "234", "235", "243", "246", "259", "263", 
"267", "271", "275", "277", "278", "287", "295", "299", "302", 
"320", "324", "326", "331", "339", "340", "344", "352", "354", 
"357", "360", "367", "368", "371", "377", "380", "381", "386", 
"392", "397", "399", "403", "404", "425", "428", "433", "437", 
"438", "441", "444", "450", "456", "465"), class = "omit"))
joemienko
  • 203
  • 1
  • 6

1 Answers1

4

Your model syntax and diagram don't match.

Where is Y13 and Y23 in your model?

Parameter C is fixed to one in your syntax, but not in the model. Is that deliberate?

Where is U?

If you could post a diagram of the model you think you are running, it makes it much easier to answer questions about it.

I think that this part i the source of your troubles:

Y11 ~~ Y12
Y21 ~~ Y22
Y11 ~~ Y22
Y21~~ Y12

I think it should be:

Y11 ~~ Y21
Y12 ~~ Y22

However, if you only have four measures, I don't think the model is going to be identified.

Edit:

In your original path diagram, there were (I think) more constraints that are not shown here. It's clearer if you add constraints to the diagram - that way we can see if the constraints you have match the constraints you think you have.

Anyway, you have 5 variables, which means you have 15 moments (5 variances, 10 covariances).

You might need to explicitly add residual variances to your latent variables.

B ~~ B
C ~~ C

Edit 2:

You need residual variances in your model. They are in the diagram, but not the model. Add:

Y_11 ~~ Y_11
Y_12 ~~ Y_12
Y_21 ~~ Y_21
Y_22 ~~ Y_22

That helps, but the model won't run with:

B ~~ X 

in there.

I suspect this is a local identification problem, but I can't quite get my head around why. The solution is more data - either more waves, or more variables within each wave.

Jeremy Miles
  • 13,917
  • 6
  • 30
  • 64
  • I apologize for the path diagram from before. I had just copied a "close" diagram to that which was shared in the presentation. I have redrawn the model I think I am trying to estimate. Do you still think it is unidentified? The presentation (from an ostensibly reputable social psychologist) appears to indicate that fixing the loadings to 1 should help with identifiability in the two indicator case. – joemienko Jun 16 '16 at 16:53
  • Note too, I have also provided a simulated set of data which produces the same error if this is helpful in providing any guidance. – joemienko Jun 16 '16 at 17:15
  • This has sufficiently answered my question here - it appears the model is overidentified. If you have any thoughts about my more simple approach here: http://stats.stackexchange.com/questions/219288/paired-t-test-as-a-simple-latent-change-score-model, it would be much appreciated. – joemienko Jun 16 '16 at 23:23