http:\/\/www.linfo.org\/stdio.html<\/a>.<\/p>\nManual pages (getting more info)<\/h3>\n
The linux manual pages are divided into sections. Section 2 is for system calls, like the ones you will see in strace output. Each system call has it\u2019s own manual page that includes all the things that are allowed to go as arguments (between the parentheses) and all the return values (things that can appear after the \u201c=\u201d sign).<\/p>\n
At this stage, the manual page is mostly useful for looking up things that appear in the strace output. For instance, an example used later will contain the following output:<\/p>\n
open(\".\/chdir.sh\", O_RDONLY) = 3<\/code><\/p>\nIf you want to know what O_RDONLY means, you can either search the web for \u201cman open linux\u201d and use your browsers \u201cfind on page\u201d function to find the information regarding O_RDONLY or you can use man 2 open and (by default) the \u201c\/\u201d key to search inside the manpage. The first instance of O_RDONLY on the page says:<\/p>\n
\nThe argument flags must include one of the following access modes: O_RDONLY, O_WRONLY, or O_RDWR. These request opening the file read-only, write-only, or read\/write, respectively.<\/p>\n<\/blockquote>\n
We now know that the file was opened read only. Lets say that instead of the above, output, we saw:<\/p>\n
open(\".\/chdir.sh\", O_RDONLY) = EACCES<\/code><\/p>\nWe then search on the manual page for EACCES and find:<\/p>\n
\nThe requested access to the file is not allowed, or search permission is denied for one of the directories in the path prefix of pathname, or the file did not exist yet and write access to the parent directory is not allowed. (See also path_resolution(7).<\/p>\n<\/blockquote>\n
This is a little verbose, but it basically says that this is an access error – like permission denied.<\/p>\n
Example: Permission Issues<\/h2>\n
Permission issues are very common on linux servers, and can sometimes be hard to track down. Strace can be a helpful tool for many permissions issues. With cPanel, you\u2019ll be using strace on a lot of perl scripts. It\u2019s also likely that there will be multiple errors in one script. This example shows two permissions and a missing file issue.<\/p>\n
\n- Create a file in your working directory called \u201copen-warn.pl\u201d and copy the contents below into it. Ensure that it is set to be executable (chmod +x open-warn.pl)<\/li>\n
- Create a file called \u201ctest-read\u201d that contains a short string, like \u201cThis is a test file\u201d, and ensure that it has 0 permissions.<\/li>\n
- Create a directory called \u201ccontent\u201d, and copy the test-read file into it.<\/li>\n
- Ensure that the content folder has 0 permissions, and that the test-read file inside the content folder it is set to 644.<\/li>\n
- Ensure that there is no file named \u201ctest-read-missing\u201d in the working directory.<\/li>\n
- Run strace on open-warn.pl, following forks and outputting to the file perl-test.strace<\/li>\n<\/ol>\n
The file for step 1:
\n[~\/public_html]# cat open-warn.pl
\n#!\/usr\/bin\/perl<\/code><\/p>\nprint “Testing bad file permissionsn”;
$filename = “test-read”;
open( FILE, “< $filename” ) or warn “Can’t open $filename : $!”;
close FILE;<\/p>\n
print “Testing bad folder permissionsn”;
$filename = “content\/test-read”;
open( FILE, “< $filename” ) or warn “Can’t open $filename : $!”;
close FILE;<\/p>\n
print “Testing missing filen”;
$filename = “test-read-missing”;
open( FILE, “< $filename” ) or warn “Can’t open $filename : $!”;
close FILE;<\/p>\n
The \u201cn\u201d at the end of each print statement tells perl to insert a newline. It also tells perl to flush the print buffer (print right now). Without them, all the error messages would print above all the print statements. (Go ahead and try it)<\/p>\n
The output from step 5:
\n[~\/public_html]# strace -f -o perl-test.strace .\/open-warn.pl
\nTesting bad file permissions
\nCan't open test-read : Permission denied at .\/open-warn.pl line 5.
\nTesting bad folder permissions
\nCan't open content\/test-read : Permission denied at .\/open-warn.pl line 10.
\nTesting missing file
\nCan't open test-read-missing : No such file or directory at .\/open-warn.pl line 15.
\n[~\/public_html]#
\n<\/code><\/p>\nSo we have two \u201cPermission Denied\u201d and one \u201cNo such file\u201d error. We can grep for either EACCES or \u201cPermission denied\u201d in the strace and find the errors quickly and easily:<\/p>\n
\n[~\/public_html]# grep EACCES ..\/perl-test.strace
\n8817 open(\"test-read\", O_RDONLY) = -1 EACCES (Permission denied)
\n8817 open(\"content\/test-read\", O_RDONLY) = -1 EACCES (Permission denied)
\n<\/code><\/p>\nWhen we grep for ENOENT or \u201cNo such file\u201d, however, we get quite a few more. Below I have two examples of pulling out some of the lines I know I don\u2019t need. Unfortunately, determining what you can safely exclude is highly context dependent.<\/p>\n
\n[~\/public_html]# grep ENOENT ..\/perl-test.strace | grep -v \"\/usr\/\"
\n8817 access(\"\/etc\/ld.so.preload\", R_OK) = -1 ENOENT (No such file or directory)
\n8817 stat(\"\/home\/mary\/perl5\/lib\/perl5\/5.10.1\/x86_64-linux-thread-multi\", 0x7fff625b1400) = -1 ENOENT (No such file or directory)
\n8817 stat(\"\/home\/mary\/perl5\/lib\/perl5\/5.10.1\", 0x7fff625b1400) = -1 ENOENT (No such file or directory)
\n8817 stat(\"\/home\/mary\/perl5\/lib\/perl5\/x86_64-linux-thread-multi\", 0x7fff625b1400) = -1 ENOENT (No such file or directory)
\n8817 stat(\"\/home\/mary\/perl5\/lib\/perl5\/5.10.0\", 0x7fff625b1400) = -1 ENOENT (No such file or directory)
\n8817 open(\"test-read-missing\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n<\/code>
or
\n[~\/public_html]# grep ENOENT ..\/perl-test.strace | grep -v \"perl5\"
\n8817 access(\"\/etc\/ld.so.preload\", R_OK) = -1 ENOENT (No such file or directory)
\n8817 open(\"\/usr\/share\/locale\/en_US.UTF-8\/LC_MESSAGES\/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n8817 open(\"\/usr\/share\/locale\/en_US.utf8\/LC_MESSAGES\/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n8817 open(\"\/usr\/share\/locale\/en_US\/LC_MESSAGES\/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n8817 open(\"\/usr\/share\/locale\/en.UTF-8\/LC_MESSAGES\/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n8817 open(\"\/usr\/share\/locale\/en.utf8\/LC_MESSAGES\/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n8817 open(\"\/usr\/share\/locale\/en\/LC_MESSAGES\/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n8817 open(\"test-read-missing\", O_RDONLY) = -1 ENOENT (No such file or directory)
\n<\/code><\/p>\nEither way, it\u2019s the last line that gives us what we\u2019re looking for.<\/p>\n
Now, this was a little easy – we knew what to look for because the error message told us. Most of the time, you\u2019ll be using strace because the error messages are not so helpful. Lets modify our open-warn.pl program and call it open-silent.pl:
\n[~\/public_html]# cat open-silent.pl
\n#!\/usr\/bin\/perl<\/code><\/p>\nprint “Testing bad file permissions”;
$filename = “test-read”;
open( FILE, “< $filename” ) or warn “Can’t open an important file”;
close FILE;<\/p>\n
print “Testing bad folder permissions”;
$filename = “content\/test-read”;
open( FILE, “< $filename” ) or warn “Can’t open an important file”;
close FILE;<\/p>\n
print “Testing missing file”;
$filename = “test-read-missing”;
open( FILE, “< $filename” ) or warn “Can’t open an important file”;
close FILE;<\/p>\n
Now when we run strace we get output like this:
\n[~\/public_html]# strace -f -o perl-quiet.strace .\/open-silent.pl
\nCan't open an important file at .\/open-silent.pl line 5.
\nCan't open an important file at .\/open-silent.pl line 10.
\nCan't open an important file at .\/open-silent.pl line 15.
\nTesting bad file permissionsTesting bad folder permissionsTesting missing [~\/public_html]#
\n<\/code><\/p>\nNow, it still gives you some hints – like the line number. But every single one of those lines reads:
open( FILE, \"< $filename\" ) or warn \"Can't open an important file\";<\/code><\/p>\nIf the program was long and complicated, we\u2019d have no way of knowing if it was the same file that was being attempted multiple times or if there were multiple files. Now the strace file tells us something useful. I would start by checking for permission errors, since they are the most common reason that a file can\u2019t be opened:
\n[~\/public_html]# grep EACCES perl-quiet.strace
\n1489 open(\"test-read\", O_RDONLY) = -1 EACCES (Permission denied)
\n1489 open(\"content\/test-read\", O_RDONLY) = -1 EACCES (Permission denied)
\n<\/code><\/p>\nThat gives us two of our three errors. Honestly, if I were debugging for real, at this point I would just check all the permissions on public_html and it\u2019s subfolders and files. Because permission issues usually come in large batches on a webserver. \ud83d\ude42<\/p>\n
Once I\u2019ve fixed the permission issues on test-read and content, I can run strace again. Now I only have one error message. I\u2019m going to check for permission issues again, because sometimes when you fix one permission issue (especially on a directory) it uncovers more permission issues. Unfortunately for me, there are no more permission issues:<\/p>\n
\n[~\/public_html]# grep EACCES perl-quiet2.strace
\n[~\/public_html]#
\n<\/code><\/p>\nSo now I have to think of what else might cause a file not to be opened. Maybe the file just isn\u2019t there. So I grep for ENOENT and man there\u2019s a lot of output. I always start at the bottom – especially if the error stops the program from running to completion. In this case, the last line is:<\/p>\n
1595 open(\"test-read-missing\", O_RDONLY) = -1 ENOENT (No such file or directory)<\/code><\/p>\nNow I know that a program related file is missing. Depending on what my actual application is depends on how I fix this. In this case, I\u2019m just going to touch the file to create it. Now when I run the script (.\/open-silent.pl) it does not output any error messages. \ud83d\ude42<\/p>\n\n\n
<\/h2>\n","protected":false},"excerpt":{"rendered":"
What is strace? Strace is a \u201csystem call trace\u201d program. It attaches to a process and tracks system calls and signals made to and from it (and possibly it\u2019s children). There are limitations to strace, some of which are detailed below. However, strace can be a very valuable tool for determining the root cause of […]<\/p>\n","protected":false},"author":77,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[61],"tags":[],"class_list":["post-21552","post","type-post","status-publish","format-standard","hentry","category-tips-and-tricks"],"acf":[],"yoast_head":"\n
Starting with Strace | cPanel<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Starting with Strace | cPanel\" \/>\n<meta property=\"og:description\" content=\"What is strace? Strace is a \u201csystem call trace\u201d program. It attaches to a process and tracks system calls and signals made to and from it (and possibly it\u2019s children). There are limitations to strace, some of which are detailed below. However, strace can be a very valuable tool for determining the root cause of […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/\" \/>\n<meta property=\"og:site_name\" content=\"cPanel\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/cpanel\/\" \/>\n<meta property=\"article:published_time\" content=\"2014-09-04T18:00:00+00:00\" \/>\n<meta name=\"author\" content=\"cPanel Community\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@cPanel\" \/>\n<meta name=\"twitter:site\" content=\"@cPanel\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"cPanel Community\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"19 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/\",\"url\":\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/\",\"name\":\"Starting with Strace | cPanel\",\"isPartOf\":{\"@id\":\"https:\/\/devel.www.cpanel.net\/#website\"},\"datePublished\":\"2014-09-04T18:00:00+00:00\",\"dateModified\":\"2014-09-04T18:00:00+00:00\",\"author\":{\"@id\":\"https:\/\/devel.www.cpanel.net\/#\/schema\/person\/8cf97408aad4fb70cf55d11a1d4f57f8\"},\"breadcrumb\":{\"@id\":\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/devel.www.cpanel.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Starting with Strace\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/devel.www.cpanel.net\/#website\",\"url\":\"https:\/\/devel.www.cpanel.net\/\",\"name\":\"cPanel\",\"description\":\"Hosting Platform of Choices\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/devel.www.cpanel.net\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/devel.www.cpanel.net\/#\/schema\/person\/8cf97408aad4fb70cf55d11a1d4f57f8\",\"name\":\"cPanel Community\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/devel.www.cpanel.net\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e1949945083b5526bb95711bd3d616b3?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e1949945083b5526bb95711bd3d616b3?s=96&d=mm&r=g\",\"caption\":\"cPanel Community\"},\"description\":\"The web hosting industry's most reliable management solution since 1997. With our first-class support and rich feature set, it's easy to see why our customers and partners make cPanel & WHM their hosting platform of choice. For more information, visit cPanel.net.\",\"sameAs\":[\"https:\/\/cpanel.net\"],\"url\":\"https:\/\/devel.www.cpanel.net\/blog\/author\/cpadmin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Starting with Strace | cPanel","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/","og_locale":"en_US","og_type":"article","og_title":"Starting with Strace | cPanel","og_description":"What is strace? Strace is a \u201csystem call trace\u201d program. It attaches to a process and tracks system calls and signals made to and from it (and possibly it\u2019s children). There are limitations to strace, some of which are detailed below. However, strace can be a very valuable tool for determining the root cause of […]","og_url":"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/","og_site_name":"cPanel","article_publisher":"https:\/\/www.facebook.com\/cpanel\/","article_published_time":"2014-09-04T18:00:00+00:00","author":"cPanel Community","twitter_card":"summary_large_image","twitter_creator":"@cPanel","twitter_site":"@cPanel","twitter_misc":{"Written by":"cPanel Community","Est. reading time":"19 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/","url":"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/","name":"Starting with Strace | cPanel","isPartOf":{"@id":"https:\/\/devel.www.cpanel.net\/#website"},"datePublished":"2014-09-04T18:00:00+00:00","dateModified":"2014-09-04T18:00:00+00:00","author":{"@id":"https:\/\/devel.www.cpanel.net\/#\/schema\/person\/8cf97408aad4fb70cf55d11a1d4f57f8"},"breadcrumb":{"@id":"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/devel.www.cpanel.net\/blog\/tips-and-tricks\/starting-with-strace\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/devel.www.cpanel.net\/"},{"@type":"ListItem","position":2,"name":"Starting with Strace"}]},{"@type":"WebSite","@id":"https:\/\/devel.www.cpanel.net\/#website","url":"https:\/\/devel.www.cpanel.net\/","name":"cPanel","description":"Hosting Platform of Choices","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/devel.www.cpanel.net\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/devel.www.cpanel.net\/#\/schema\/person\/8cf97408aad4fb70cf55d11a1d4f57f8","name":"cPanel Community","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/devel.www.cpanel.net\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e1949945083b5526bb95711bd3d616b3?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e1949945083b5526bb95711bd3d616b3?s=96&d=mm&r=g","caption":"cPanel Community"},"description":"The web hosting industry's most reliable management solution since 1997. With our first-class support and rich feature set, it's easy to see why our customers and partners make cPanel & WHM their hosting platform of choice. For more information, visit cPanel.net.","sameAs":["https:\/\/cpanel.net"],"url":"https:\/\/devel.www.cpanel.net\/blog\/author\/cpadmin\/"}]}},"_links":{"self":[{"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/posts\/21552"}],"collection":[{"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/users\/77"}],"replies":[{"embeddable":true,"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/comments?post=21552"}],"version-history":[{"count":0,"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/posts\/21552\/revisions"}],"wp:attachment":[{"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/media?parent=21552"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/categories?post=21552"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devel.www.cpanel.net\/wp-json\/wp\/v2\/tags?post=21552"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}