Showing the last text line from a file in QC, how?

pibo's picture

Hi to all,

I need to show in a QC movie just the last line of text* from the latest .log** file inside a local directory.

(*)File name structure:

127.0.0.1,[number].log

Note: inside the specified /Directory are all the .log files, the latest is the one which its higher value in the [number] file name part.

(**)New line format inside de .log:

[D/M/Y h:m:s] * Text for showing

Note: I don´t want to display the date and the *, just the "Text for showing" part

Thank you very much for your help.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi again,

I missed something: The log is constantly updated in real time, so I need to show in QC the text from the last line just when it changes.

Thank you.

photonal's picture
Re: Showing the last text line from a file in QC, how?

There is a Quartz Composer plugin Apple Developer example called CommandLineTool which can be used to execute unix commands. I used it together with a shell script to 'sed' the log file and remove characters up until and including when the '*' occurs.

Source Code of the plugin here:

http://developer.apple.com/library/mac/#samplecode/CommandLineTool/Intro...

It works by regularly 'tail'ing the last last line of the log file and executing the script.sh script on it; which contains the sed command to remove the leading characters to the '*'. Finally the output is written to screen (together with the original output) - if any errors occur they are output at the bottom of the screen.

I had originally wanted to place the sed processing in the CommandLine tool chain - but couldn't get it to work; hence the script calling workaround.

The attached zip archive contains

i) compiled plugin (place CommandLineTool.plugin in your /Library/Graphics/Quartz Composer Patches folder)

ii) bash script (make a note of the path where you save it, e.g. /Users/USERNAME/Desktop/script.sh if you save it to your desktop)

iii) CommandLineTool-bash-script-1.qtz (remember to change the PATH in Argument #2 of "Get the last line of the file" to your log file and enter the path of where you saved the script.sh file in "Call script")

Hope it helps!

P.S:. and make sure the file permissions on script.sh are 744 so that it will execute. (run 'chmod 744 script.sh' in a terminal)

PreviewAttachmentSize
CommandLine.zip23.1 KB

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi photonal,

Thank you so much for your response. I am so sorry but I haven´t found how to run your files.

i) compiled plugin (place CommandLineTool.plugin in your /Library/Graphics/Quartz Composer Patches folder)

DONE: I have placed it in:

Macintosh HD/ Library/ Graphics/ Quartz Composer Patches/ CommandLineTool.plugin

ii) bash script (make a note of the path where you save it, e.g. /Users/USERNAME/Desktop/script.sh if you save it to your desktop)

DONE: I have placed it in:

Users/ Me/ script.sh

iii) CommandLineTool-bash-script-1.qtz

¿...? I have opened CommandLineTool-bash-script-1.qtz in QC4, then I can see the attached structure. That´s one of my doubts:

(remember to change the PATH in Argument #2 of "Get the last line of the file" to your log file and enter the path of where you saved the script.sh file in "Call script")

I don´t see "Get the last line of the file" and of obviously, the path argument inside.

How can I see "Get the last line of the file"?

PS: I ran in Terminal "chmod 744 /Users/Me/script.sh" before that.

You have explained so well, maybe it´s me that I don´t understand any of your instructions. I wish you could help me out with my issues about your solution.

Thank you so much again.

PreviewAttachmentSize
capt.png
capt.png155.45 KB

gtoledo3's picture
Re: Showing the last text line from a file in QC, how?

I don't have a computer handy to look at the files on, but I do know that the original command line plugin needs to be installed in a plugin folder, not patches.

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi,

I have changed it to the plugin folder and it has worked, somehow... Let me show you in the attached caption the output from the viewer.

The data is shown (the example last line text is "Arcade Fire - No Cars Go"), but for some reason the output shows a lot of info in 2 lines: Original Input and Output.

As you can see in the image, there are two sprites (I have formatted them first for showing all the contents).

I have 2 questions:

[1] "Test the '*'" has:

Path argument: /bin/echo

and

Argument#1: 2010-11-05 17:03:03.000 Xcode[828:7903]: device disconnected (isDFU = 1)*1

Do I have to change anything here?

[2] "Get the last line of the file"

I have changed Argument #2 with the path of my .log file

but Path: /usr/bin/tail, should it be changed also?


The "Call script" has the correct Path to the script.sh

AT THIS MOMENT...

GOOD:

  • Somehow, it is shown the last line text.

  • It updates in realtime when there is a new line inside the .log

  • Somehow, it breaks the line for the "*" character.

WRONG:

  • Two lines (input/output) are shown, with a lot of info before the expected text.

  • You must enter manually inside the movie, the name of the last .log file (remember there are a lot of .log files inside the same directory, the newer, the one which has the higher number in its file name).

  • UTF-8 characters are not shown and breaks the rest of the text (i.e. "Canción" is shown like "Canci")

Thank you again.

PreviewAttachmentSize
capt.png
capt.png52.45 KB

photonal's picture
Re: Showing the last text line from a file in QC, how?

ooops, sorry about that - I hadn't noticed that I had a duplicate of the CommandLineTool plugin in the 'Plug-ins' folder too; which was why I thought the one in the patches folder was working ;-)

Yes, the Plug-ins folder is where it should be installed.

In answer to your questions:

1) The "Test the '*'", is just a patch to check that the text snip after the asterisk was working, so no need to change anything there.

2) In "Get the last line of the file", there is no need to change the path of tail either. I only included the paths to the unix commands because there were times when they wouldn't work unless I included the full path.

Could you provide a couple examples of the log files? Perhaps edit them to just include the last ten lines (preferably with some UTF-8 characters in them) of each if they are very large.

PreviewAttachmentSize
CommandLineTool.png
CommandLineTool.png178.22 KB

photonal's picture
Re: Showing the last text line from a file in QC, how?

pibo wrote:
...The data is shown (the example last line text is "Arcade Fire - No Cars Go"), but for some reason the output shows a lot of info in 2 lines: Original Input and Output.

Looks like your whole / (root) directory is being processed there, check that you don't have a space after a '/' in the paths you added.

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi again,

I have attached 2 consecutive .log files as an example with special characters entries. These are the .logs generated from the app, as they are, without editing anything. I have tried again in QC, and it shows the special characters in the viewer, DONE.

Now, the 2 left issues are:

[1] I have noticed the * stripping doesn´t work for me (the * makes that the "Applications Developer Library..." large sentence appears).

  • Instead of breaking the line from the * character (at this moment, I couldn´t make it work, it is shown the whole line), it would be done by stripping it from the 24th character also, maybe changing the sed 's/.**//' in the script.sh file.

[2] How to autoload the latest .log file in QC?

  • The .log file name is different each time a new one is created, just in the "NUMBER" part: "127.0.0.1,NUMBER.log" (the newer the file, the higher the NUMBER is).

Notes:

  • About the directory /Users/Me/ and the structure and also the name of the .log files, they can´t be changed.

Thank you so much again.

PreviewAttachmentSize
example-logs.zip766 bytes

pibo's picture
Re: Showing the last text line from a file in QC, how?

photonal wrote:
pibo wrote:
...The data is shown (the example last line text is "Arcade Fire - No Cars Go"), but for some reason the output shows a lot of info in 2 lines: Original Input and Output.

Looks like your whole / (root) directory is being processed there, check that you don't have a space after a '/' in the paths you added.

You are right, the .logs are stored at root, but the space after the '/' was an issue of this transcription, it´s ok in the argument fields.

photonal's picture
Re: Showing the last text line from a file in QC, how?

This 'asterisk' in the log files is causing this. Any chance to write the logs without the 'asterisk's? Otherwise the logs will have to be pre-processed to remove them in this current workaround for it to work correctly.

photonal's picture
Re: Showing the last text line from a file in QC, how?

This version (as before change the paths in the various patches for your system) will process new log files as they are created, however I haven't found a way to get around the asterisk problem - they will need pre-processing to get rid of it otherwise the asterisk gets used as a parameter which is why the extra Directories turn up in the output data.

Anyway, hope this helps as a building block to getting what you want.

pibo's picture
Re: Showing the last text line from a file in QC, how?

photonal wrote:
This version (as before change the paths in the various patches for your system) will process new log files as they are created

Even if they have new names? I have tried it with several .logs in the same directory, and it always loads the specified file in Argument #2, not the newest file with new file name.


Another issue is when it´s running for a while the background and the text disappear from the Viewer (it´s like it crashes or something). Then, when clicking Stop and Run, the background and the text come back again but the text scale changes dramatically in the Viewer, except in the sprite Width and Height parameters.

Thank you.

photonal's picture
Re: Showing the last text line from a file in QC, how?

Here's the new version which reads in new log files.

Yeah the text does go haywire after a while - no idea why that id happening.

PreviewAttachmentSize
CommandLineTool-2.zip5.06 KB

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi,

I have tried CommandLineTool-2 with no results. I have changed before running it the paths in your files and tried from /Users/Me/ and Users/Me/Desktop/ (with the .log files in these directories in both cases) having the same output in the Viewer: "Original Input: (blank)" "Output: (blank)". The .sh files are 744 already also.

Did it worked for you with my .log files that I have attached in a previous post?

About the crashes, If there is no way to arrange them, it makes the script useless actually.

Thank you again.

photonal's picture
Re: Showing the last text line from a file in QC, how?

This is the output from the 2nd version.

But like I mentioned, some preprocessing would be needed to be done to the log file to remove the asterisks first; could be done with a script using the 'sed' unix command. Hopefully there are pointers here to help you complete what you are looking for.

PreviewAttachmentSize
Output.jpg
Output.jpg64.52 KB

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi,

As I said you before, I can´t get anything in the viewer. I think no .log file is loaded (or something like that), in my case with version 2. How have you done that?

Let me know what I need to change in your files, in my case I put them at /Users/Me/ (in the same directory with the .logs), and changed the directories inside the CommandLineTool-bash-script-2

Do I have to make more changes than these? Do you mean I need to remove the * inside the logs? It can´t be done as they come directly from the software.

Another thing is that I have ran it in Profile mode, and I see the crash issue could be in the LFO settings (when disconnecting it, the script doesn´t crash). Is it necessary to have it as fast as you have configured it?

Thank you.

photonal's picture
Re: Showing the last text line from a file in QC, how?

Ok, hopefully this one does a better job.

It assumes your path is : '/Users/pibo/Desktop/'; change as necessary (change in 'Get latest log file', 'Get the last line of the file' and in the scripts too.)

It also assumes your log files are being written to your desktop.

i) it looks for new log files less frequently (set changing the LFO Period)

ii) it pre-processes the log file to remove the first 24 characters (and the offending asterisk)

PreviewAttachmentSize
Output-3.jpg
Output-3.jpg29.35 KB
CommandLineTool-3.zip5.17 KB

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi again,

It seems that the crashes are gone, but now there is something unexpected in its behaviour now (at least for me):

I have in Desktop the "Scripts" folder, the "CommandLineTool-bash-script-3" file and this 2 .logs also:

127.0.0.1,50454.log

  • [06-11-2010 12:39:11] Log file created for client "127.0.0.1,50454". [06-11-2010 12:39:11] Client "127.0.0.1,50454" connected and has been identified as Traktor (or a something close enough). [06-11-2010 12:39:34] * Stateless - Prism #1 (Prince Language Remix) [06-11-2010 12:40:06] * Bot'Ox - Blue Steel [06-11-2010 12:40:29] * Maga - Año Nuevo [06-11-2010 12:41:21] * Lori Meyers - Hostal Pimodán

127.0.0.1,50604.log

  • [04-11-2010 15:27:01] Log file created for client "127.0.0.1,50604". [04-11-2010 15:27:01] Client "127.0.0.1,50604" connected and has been identified as Traktor (or a something close enough). [04-11-2010 15:27:27] * Alan Braxe & Fred Falke - Rubicon [04-11-2010 15:27:46] * Arcade Fire - No Cars Go

As the "127.0.0.1,50604.log" file is the last one, it is supposed that the Viewer shows "Arcade Fire - No Cars Go", but:

The first time, it is shown: "Lori Meyers - Hostal Pimodán" The 2nd: "odán" ...

Explanation:

The first time your function runs, it loads the 1st .log file (not the last one) and creates a new "127.0.0.1,50454.log.log" (2 .log) with just the last line of the previous file without the first 24 characters. So it is "Lori Meyers - Hostal Pimodán" in it, and shows this line in the Viewer.

The second time it passes, it loads the "127.0.0.1,50454.log.log" file (not the last one) and creates a new "127.0.0.1,50454.log.log.log" (3 .log) with just the last line of the previous file without the first 24 characters. So it is: "odán" in it, and shows this line in the Viewer.

(etc)

I don´t know how to work with sed, so I can´t find how to help and arrange this by myself, but don´t worry if you don´t find any solution that works. It´s a pity, but maybe it is not possible to make it work correctly as expected.

Anyway, thank you so much for all your work.

photonal's picture
Re: Showing the last text line from a file in QC, how?

Well the period of the LFO could be tweaked to match more closely the rate at which the log files are being created.

btw, there's plenty of 'sed' documentation online.

Anyway, good luck with your project!

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi again,

I can have the SED code working at 100% right now (thanks Photonal for your previous help here).

Both .sh files are now:

[getFileName.sh]

#!/bin/bash cd /Users/Me/ ls -tr *.log | tail -1

[getLastLineOfLog.sh]

#!/bin/bash sed -ne'$s/^.{24}//p' /Users/Me/$1

BUT there are critical issues now with Quart Composer in [CommandLineTool-bash-script-3]:

  • The script still crashes after a time (about 30 sec, 1 min, whatever).
  • After crashing and running again, the text is showed in the Viewer in a random scale.
  • The script doesn´t autoload and autoupdate both files and the last lines. You must do it by stopping and running again the script.

Hope I could find a solution for this. Maybe is the LFO and Pulse, that don´t work as expected.

pibo's picture
Re: Showing the last text line from a file in QC, how?

pibo wrote:
There are critical issues now with Quart Composer in [CommandLineTool-bash-script-3]:

  • The script still crashes after a time (about 30 sec, 1 min, whatever).
  • After crashing and running again, the text is showed in the Viewer in a random scale.

AS Smokris (developer) said in another forum, maybe it is a bug in the current version of the Command Line Tool.

photonal's picture
Re: Showing the last text line from a file in QC, how?

Hi Pibo,

I tried out your version with an LFO triggering the scripts and it is working fine without any crashes or text scaling issues.

PreviewAttachmentSize
LastTrackTiggered.qtz4.06 KB

pibo's picture
Re: Showing the last text line from a file in QC, how?

Hi again photonal,

I thought for a moment it has been done already, but the LastTrackTiggered.qtz crashed after 2 minutes, and when I runned it again from the Viewer, the random scale appears and no data has been reloaded anymore.

Things go normal for a while when I restart QC with LastTrackTiggered again, but then it crashes suddenly as usual.

Have you tried it out for about 2-5 minutes entering new lines in the current .log on the fly and has it worked? Let me know, please.

Regards.

pibo's picture
Re: Showing the last text line from a file in QC, how?

Does anybody know why this LastTrack composition crashes after a short time running?

I have tried unplugging the LFO/Pulse and Enable/Disable by hand continuosly both Command Line Tool patches.

When doing it in "Get lastest .log file" patch, it outputted this error:

An exception was raised:

*** -[NSConcreteTask terminationStatus]: task still running

0x85b9bc55: -[QCContext nextExecutionTimeForPatch:time:arguments:] 0x85b9b9fe: -[QCGraphicsContext nextExecutionTimeForPatch:time:arguments:] 0x85b9b7e1: -[QCOpenGLContext nextExecutionTimeForPatch:time:arguments:] 0x0000d5f2 0x85be372d: -[QCView render:arguments:] 0x84ad6c49: -[NSView _drawRect:clip:] 0x84ad58bc: -[NSView _recursiveDisplayAllDirtyWithLockFocus:visRect:] 0x84ad5c26: -[NSView _recursiveDisplayAllDirtyWithLockFocus:visRect:] 0x84ad3f8e: -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad4e5a: -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad4e5a: -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad3ab0: -[NSThemeFrame _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad0362: -[NSView _displayRectIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:] 0x84a49b9a: -[NSView displayIfNeeded] 0x84a44a46: _handleWindowNeedsDisplay 0x85471b37: __CFRunLoopDoObservers 0x8544d464: __CFRunLoopRun 0x8544cdbf: CFRunLoopRunSpecific 0x8470591a: RunCurrentEventLoopInMode 0x8470567d: ReceiveNextEventCommon 0x847055d8: BlockUntilNextEventMatchingListInMode 0x84a19e64: _DPSNextEvent 0x84a197a9: -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] 0x849df48b: -[NSApplication run]

When doing it in "Get the last line of the .log file" patch, it outputted this error (i think is the same):

An exception was raised:

*** -[NSConcreteTask terminationStatus]: task still running

0x85b9bc55: -[QCContext nextExecutionTimeForPatch:time:arguments:] 0x85b9b9fe: -[QCGraphicsContext nextExecutionTimeForPatch:time:arguments:] 0x85b9b7e1: -[QCOpenGLContext nextExecutionTimeForPatch:time:arguments:] 0x0000d5f2 0x85be372d: -[QCView render:arguments:] 0x84ad6c49: -[NSView _drawRect:clip:] 0x84ad58bc: -[NSView _recursiveDisplayAllDirtyWithLockFocus:visRect:] 0x84ad5c26: -[NSView _recursiveDisplayAllDirtyWithLockFocus:visRect:] 0x84ad3f8e: -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad4e5a: -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad4e5a: -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad3ab0: -[NSThemeFrame _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] 0x84ad0362: -[NSView _displayRectIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:] 0x84a49b9a: -[NSView displayIfNeeded] 0x84a44a46: _handleWindowNeedsDisplay 0x85471b37: __CFRunLoopDoObservers 0x8544d464: __CFRunLoopRun 0x8544cdbf: CFRunLoopRunSpecific 0x8470591a: RunCurrentEventLoopInMode 0x8470567d: ReceiveNextEventCommon 0x847055d8: BlockUntilNextEventMatchingListInMode 0x84a19e64: _DPSNextEvent 0x84a197a9: -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] 0x849df48b: -[NSApplication run]

Thank you in advance for your help.

PreviewAttachmentSize
LastTrack.zip31.56 KB